Page 1
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Explaining Optimization inGenetic Algorithms with Uniform Crossover
Keki Burjorjeekekibcsbrandeisedu
Foundations of Genetic Algorithms Conference XII (2013)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Crossover
Uniform Crossover first used by Ackley (1987)
Studied and popularized by Syswerda (1989) and Spears amp DeJong (1990 1991)
Numerous accounts of optimization in GAs with UX
In practice frequently outperforms XO with tight linkage(Fogel 2006)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Crossover
Uniform Crossover first used by Ackley (1987)
Studied and popularized by Syswerda (1989) and Spears amp DeJong (1990 1991)
Numerous accounts of optimization in GAs with UX
In practice frequently outperforms XO with tight linkage(Fogel 2006)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Crossover
Uniform Crossover first used by Ackley (1987)
Studied and popularized by Syswerda (1989) and Spears amp DeJong (1990 1991)
Numerous accounts of optimization in GAs with UX
In practice frequently outperforms XO with tight linkage(Fogel 2006)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Crossover
Uniform Crossover first used by Ackley (1987)
Studied and popularized by Syswerda (1989) and Spears amp DeJong (1990 1991)
Numerous accounts of optimization in GAs with UX
In practice frequently outperforms XO with tight linkage(Fogel 2006)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Optimization in GAs with UX Unexplained
Cannot be explained within the rubric of the BBH
No viable alternative has been proposed
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Optimization in GAs with UX Unexplained
Hyperclimbing Hypothesis
A scientific explanation for optimization in GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Does ldquoScientificrdquo Mean
Logical Positivism (Proof or Bust)
Scientific truth is absolute
Emphasis on Verifiability (ie mathematical proof)
The Popperian method
Scientific truth is provisional
Emphasis on Falsifiability (testable predictions)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Does ldquoScientificrdquo Mean
Logical Positivism (Proof or Bust)
Scientific truth is absolute
Emphasis on Verifiability (ie mathematical proof)
The Popperian method
Scientific truth is provisional
Emphasis on Falsifiability (testable predictions)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Popperian Method
Logic behind the Popperian Method Contrapositive
Theory rArr Phenomenon hArr notPhenomenonrArr notTheory
Additional tightening by Popper
Unexpected Phenomenon rarr More credence owed to the theoryeg Gravitational lensing rarr General theory of relativity
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Popperian Method
Logic behind the Popperian Method Contrapositive
Theory rArr Phenomenon hArr notPhenomenonrArr notTheory
Additional tightening by Popper
Unexpected Phenomenon rarr More credence owed to the theory
eg Gravitational lensing rarr General theory of relativity
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Additional Requirements in a Science of EC
Weak assumptions about distribution of fitness
This is just Occamrsquos Razor
Upfront proof of concept
Avoid another rdquoRoyal Roads momentrdquo
(Nice to have) Identification of a core computational efficiency
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Additional Requirements in a Science of EC
Weak assumptions about distribution of fitness
This is just Occamrsquos Razor
Upfront proof of concept
Avoid another rdquoRoyal Roads momentrdquo
(Nice to have) Identification of a core computational efficiency
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Additional Requirements in a Science of EC
Weak assumptions about distribution of fitness
This is just Occamrsquos Razor
Upfront proof of concept
Avoid another rdquoRoyal Roads momentrdquo
(Nice to have) Identification of a core computational efficiency
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Hypothesis
GAs with UX perform optimization by efficientlyimplementing a global search heuristic called hyperclimbing
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Outline
1 Highlight Symmetry of UX
2 Describe Hyperclimbing Heuristic
3 Provide proof-of-concept
Show a GA implementing hyperclimbing efficiently
4 Make a prediction and validate it on
MAX-3SATSherrington Kirkpatrick Spin Glasses problem
5 Outline future work
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0
1 0 0 1 1 0 0 1 1 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0X1 X2 X3 X4 X5 X6 X7 X8 X9 X`1 0 0 1 1 0 0 1 1 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0uarr darr darr uarr uarr darr darr darr uarr uarr1 0 0 1 1 0 0 1 1 1
0 0 0 0 1 0 0 1 1 0
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0uarr darr darr uarr uarr darr darr darr uarr uarr1 0 0 1 1 0 0 1 1 1
0 0 0 0 1 0 0 1 1 0Y1 Y2 Y3 Y4 Y5 Y6 Y7 Y8 Y9 Y`
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0uarr darr darr uarr uarr darr darr darr uarr uarr1 0 0 1 1 0 0 1 1 1
0 0 0 0 1 0 0 1 1 0dArr dArr dArr z dArr dArr dArr z dArr dArr0 0 0 1 1 0 0 0 1 0
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
X1 X2 X3 X4 X5 X6 X7 X8 X9 X`
Y1 Y2 Y3 Y4 Y5 Y6 Y7 Y8 Y9 Y`
UX X1 X` are independent
Absence of positional bias (Eshelman et al 1989)
Attributes can be arbitrarily permuted
Suppose Y1 Y` independent and independent of `
If locus i immaterial to fitness can be spliced out
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
The Hyperclimbing Heuristic
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing A Stochastic Search Heuristic
The Setting
0 1` frarr R
f may be stochastic
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing A Stochastic Search Heuristic
The Setting
0 1` frarr R
f may be stochastic
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
The Effect of a Schema Partition
Effect of JIK Variance of average fitness of schemata in JIK
If I sub I prime then Effect(I) le Effect(I prime)
JIK
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
The Effect of a Schema Partition
Effect of JIK Variance of average fitness of schemata in JIK
If I sub I prime then Effect(I) le Effect(I prime)
JIK JI primeK
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption is Weak
Staggered coarse conditional effects assumption is weakWeaker than fitness distribution assumption in the BBH
BBH assumes unstaggered coarse unconditional effects
Weaker assumptions are more likely to be satisfied in practice
Good bc we aspire to explain optimization in all GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption is Weak
Staggered coarse conditional effects assumption is weakWeaker than fitness distribution assumption in the BBH
BBH assumes unstaggered coarse unconditional effects
Weaker assumptions are more likely to be satisfied in practice
Good bc we aspire to explain optimization in all GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Proof Of Concept
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
fitness(x) simN (+025 1) if xi1 oplus xi2 oplus xi3 oplus xi4 = 1N (minus025 1) otherwise
0 Fitness
Pro
babi
lity
minus025 +025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
darr darr darr darr10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
uarr uarr uarr uarrEffective Loci
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Naive approach
Visit loci in groups of four
Check for differentiation inthe multivariate marginalfitness values
Time complexity is Ω(`4)
xi1 xi2 xi3 xi4
ExpectedMarginalFitness
0 0 0 0 minus0250 0 0 1 +0250 0 1 0 +0250 0 1 1 minus0250 1 0 0 +0250 1 0 1 minus0250 1 1 0 minus0250 1 1 1 +0251 0 0 0 +0251 0 0 1 minus0251 0 1 0 minus0251 0 1 1 +0251 1 0 0 minus0251 1 0 1 +0251 1 1 0 +0251 1 1 1 minus025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Visiting loci in groups ofthree or less wonrsquot work
xj
ExpectedMarginalFitness
0 001 00
xj1 xj2
ExpectedMarginalFitness
0 0 000 1 001 0 001 1 00
xj1 xj2 xj3
ExpectedMarginalFitness
0 0 0 000 0 1 000 1 0 000 1 1 001 0 0 001 0 1 001 1 0 001 1 1 00
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=200
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity200AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Symmetry Analytic Conclusion
Expected generations for the red dots to diverge constant wrt
Location of the red dots
Number of blue dots
Dynamics of a blue dot invariant to
Location of the red dots
Number of blue dots
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=1000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity1000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=10000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity10000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
Staircase function descriptor (m n δ σ ` LV )
L =
3 87 middot middot middot 93
42 58 middot middot middot 72
67 73 middot middot middot 81
︸ ︷︷ ︸
n
m V =
1 0 middot middot middot 11 0 middot middot middot 0
0 1 middot middot middot 1
︸ ︷︷ ︸
n
m
Staircase function is a stochastic function f 0 1` rarr R
0 δ 2δ (mminus1)δ mδ
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
δ = 02 σ = 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredInOrder__ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
trials=100 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
35 25 37 4631 5 32 2021 33 50 4240 27 30 312 14 48 2
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
B
Watch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
0 0 1 01 1 0 01 1 0 11 1 0 10 0 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=randommpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Prediction amp Validation
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
crosstype=2Mut=003Animmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
ctype=2StaircaseClampingAnimmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60000
100
200
300
400
500
600
700
800
900
Generation
Un
mu
tate
d lo
ci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
100
200
300
400
500
600
700
800
900
1000
Generation
Unm
uta
ted loci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Conclusion
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 2
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Crossover
Uniform Crossover first used by Ackley (1987)
Studied and popularized by Syswerda (1989) and Spears amp DeJong (1990 1991)
Numerous accounts of optimization in GAs with UX
In practice frequently outperforms XO with tight linkage(Fogel 2006)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Crossover
Uniform Crossover first used by Ackley (1987)
Studied and popularized by Syswerda (1989) and Spears amp DeJong (1990 1991)
Numerous accounts of optimization in GAs with UX
In practice frequently outperforms XO with tight linkage(Fogel 2006)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Crossover
Uniform Crossover first used by Ackley (1987)
Studied and popularized by Syswerda (1989) and Spears amp DeJong (1990 1991)
Numerous accounts of optimization in GAs with UX
In practice frequently outperforms XO with tight linkage(Fogel 2006)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Crossover
Uniform Crossover first used by Ackley (1987)
Studied and popularized by Syswerda (1989) and Spears amp DeJong (1990 1991)
Numerous accounts of optimization in GAs with UX
In practice frequently outperforms XO with tight linkage(Fogel 2006)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Optimization in GAs with UX Unexplained
Cannot be explained within the rubric of the BBH
No viable alternative has been proposed
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Optimization in GAs with UX Unexplained
Hyperclimbing Hypothesis
A scientific explanation for optimization in GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Does ldquoScientificrdquo Mean
Logical Positivism (Proof or Bust)
Scientific truth is absolute
Emphasis on Verifiability (ie mathematical proof)
The Popperian method
Scientific truth is provisional
Emphasis on Falsifiability (testable predictions)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Does ldquoScientificrdquo Mean
Logical Positivism (Proof or Bust)
Scientific truth is absolute
Emphasis on Verifiability (ie mathematical proof)
The Popperian method
Scientific truth is provisional
Emphasis on Falsifiability (testable predictions)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Popperian Method
Logic behind the Popperian Method Contrapositive
Theory rArr Phenomenon hArr notPhenomenonrArr notTheory
Additional tightening by Popper
Unexpected Phenomenon rarr More credence owed to the theoryeg Gravitational lensing rarr General theory of relativity
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Popperian Method
Logic behind the Popperian Method Contrapositive
Theory rArr Phenomenon hArr notPhenomenonrArr notTheory
Additional tightening by Popper
Unexpected Phenomenon rarr More credence owed to the theory
eg Gravitational lensing rarr General theory of relativity
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Additional Requirements in a Science of EC
Weak assumptions about distribution of fitness
This is just Occamrsquos Razor
Upfront proof of concept
Avoid another rdquoRoyal Roads momentrdquo
(Nice to have) Identification of a core computational efficiency
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Additional Requirements in a Science of EC
Weak assumptions about distribution of fitness
This is just Occamrsquos Razor
Upfront proof of concept
Avoid another rdquoRoyal Roads momentrdquo
(Nice to have) Identification of a core computational efficiency
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Additional Requirements in a Science of EC
Weak assumptions about distribution of fitness
This is just Occamrsquos Razor
Upfront proof of concept
Avoid another rdquoRoyal Roads momentrdquo
(Nice to have) Identification of a core computational efficiency
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Hypothesis
GAs with UX perform optimization by efficientlyimplementing a global search heuristic called hyperclimbing
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Outline
1 Highlight Symmetry of UX
2 Describe Hyperclimbing Heuristic
3 Provide proof-of-concept
Show a GA implementing hyperclimbing efficiently
4 Make a prediction and validate it on
MAX-3SATSherrington Kirkpatrick Spin Glasses problem
5 Outline future work
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0
1 0 0 1 1 0 0 1 1 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0X1 X2 X3 X4 X5 X6 X7 X8 X9 X`1 0 0 1 1 0 0 1 1 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0uarr darr darr uarr uarr darr darr darr uarr uarr1 0 0 1 1 0 0 1 1 1
0 0 0 0 1 0 0 1 1 0
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0uarr darr darr uarr uarr darr darr darr uarr uarr1 0 0 1 1 0 0 1 1 1
0 0 0 0 1 0 0 1 1 0Y1 Y2 Y3 Y4 Y5 Y6 Y7 Y8 Y9 Y`
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0uarr darr darr uarr uarr darr darr darr uarr uarr1 0 0 1 1 0 0 1 1 1
0 0 0 0 1 0 0 1 1 0dArr dArr dArr z dArr dArr dArr z dArr dArr0 0 0 1 1 0 0 0 1 0
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
X1 X2 X3 X4 X5 X6 X7 X8 X9 X`
Y1 Y2 Y3 Y4 Y5 Y6 Y7 Y8 Y9 Y`
UX X1 X` are independent
Absence of positional bias (Eshelman et al 1989)
Attributes can be arbitrarily permuted
Suppose Y1 Y` independent and independent of `
If locus i immaterial to fitness can be spliced out
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
The Hyperclimbing Heuristic
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing A Stochastic Search Heuristic
The Setting
0 1` frarr R
f may be stochastic
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing A Stochastic Search Heuristic
The Setting
0 1` frarr R
f may be stochastic
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
The Effect of a Schema Partition
Effect of JIK Variance of average fitness of schemata in JIK
If I sub I prime then Effect(I) le Effect(I prime)
JIK
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
The Effect of a Schema Partition
Effect of JIK Variance of average fitness of schemata in JIK
If I sub I prime then Effect(I) le Effect(I prime)
JIK JI primeK
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption is Weak
Staggered coarse conditional effects assumption is weakWeaker than fitness distribution assumption in the BBH
BBH assumes unstaggered coarse unconditional effects
Weaker assumptions are more likely to be satisfied in practice
Good bc we aspire to explain optimization in all GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption is Weak
Staggered coarse conditional effects assumption is weakWeaker than fitness distribution assumption in the BBH
BBH assumes unstaggered coarse unconditional effects
Weaker assumptions are more likely to be satisfied in practice
Good bc we aspire to explain optimization in all GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Proof Of Concept
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
fitness(x) simN (+025 1) if xi1 oplus xi2 oplus xi3 oplus xi4 = 1N (minus025 1) otherwise
0 Fitness
Pro
babi
lity
minus025 +025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
darr darr darr darr10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
uarr uarr uarr uarrEffective Loci
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Naive approach
Visit loci in groups of four
Check for differentiation inthe multivariate marginalfitness values
Time complexity is Ω(`4)
xi1 xi2 xi3 xi4
ExpectedMarginalFitness
0 0 0 0 minus0250 0 0 1 +0250 0 1 0 +0250 0 1 1 minus0250 1 0 0 +0250 1 0 1 minus0250 1 1 0 minus0250 1 1 1 +0251 0 0 0 +0251 0 0 1 minus0251 0 1 0 minus0251 0 1 1 +0251 1 0 0 minus0251 1 0 1 +0251 1 1 0 +0251 1 1 1 minus025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Visiting loci in groups ofthree or less wonrsquot work
xj
ExpectedMarginalFitness
0 001 00
xj1 xj2
ExpectedMarginalFitness
0 0 000 1 001 0 001 1 00
xj1 xj2 xj3
ExpectedMarginalFitness
0 0 0 000 0 1 000 1 0 000 1 1 001 0 0 001 0 1 001 1 0 001 1 1 00
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=200
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity200AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Symmetry Analytic Conclusion
Expected generations for the red dots to diverge constant wrt
Location of the red dots
Number of blue dots
Dynamics of a blue dot invariant to
Location of the red dots
Number of blue dots
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=1000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity1000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=10000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity10000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
Staircase function descriptor (m n δ σ ` LV )
L =
3 87 middot middot middot 93
42 58 middot middot middot 72
67 73 middot middot middot 81
︸ ︷︷ ︸
n
m V =
1 0 middot middot middot 11 0 middot middot middot 0
0 1 middot middot middot 1
︸ ︷︷ ︸
n
m
Staircase function is a stochastic function f 0 1` rarr R
0 δ 2δ (mminus1)δ mδ
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
δ = 02 σ = 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredInOrder__ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
trials=100 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
35 25 37 4631 5 32 2021 33 50 4240 27 30 312 14 48 2
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
B
Watch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
0 0 1 01 1 0 01 1 0 11 1 0 10 0 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=randommpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Prediction amp Validation
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
crosstype=2Mut=003Animmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
ctype=2StaircaseClampingAnimmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60000
100
200
300
400
500
600
700
800
900
Generation
Un
mu
tate
d lo
ci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
100
200
300
400
500
600
700
800
900
1000
Generation
Unm
uta
ted loci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Conclusion
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 3
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Crossover
Uniform Crossover first used by Ackley (1987)
Studied and popularized by Syswerda (1989) and Spears amp DeJong (1990 1991)
Numerous accounts of optimization in GAs with UX
In practice frequently outperforms XO with tight linkage(Fogel 2006)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Crossover
Uniform Crossover first used by Ackley (1987)
Studied and popularized by Syswerda (1989) and Spears amp DeJong (1990 1991)
Numerous accounts of optimization in GAs with UX
In practice frequently outperforms XO with tight linkage(Fogel 2006)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Crossover
Uniform Crossover first used by Ackley (1987)
Studied and popularized by Syswerda (1989) and Spears amp DeJong (1990 1991)
Numerous accounts of optimization in GAs with UX
In practice frequently outperforms XO with tight linkage(Fogel 2006)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Optimization in GAs with UX Unexplained
Cannot be explained within the rubric of the BBH
No viable alternative has been proposed
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Optimization in GAs with UX Unexplained
Hyperclimbing Hypothesis
A scientific explanation for optimization in GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Does ldquoScientificrdquo Mean
Logical Positivism (Proof or Bust)
Scientific truth is absolute
Emphasis on Verifiability (ie mathematical proof)
The Popperian method
Scientific truth is provisional
Emphasis on Falsifiability (testable predictions)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Does ldquoScientificrdquo Mean
Logical Positivism (Proof or Bust)
Scientific truth is absolute
Emphasis on Verifiability (ie mathematical proof)
The Popperian method
Scientific truth is provisional
Emphasis on Falsifiability (testable predictions)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Popperian Method
Logic behind the Popperian Method Contrapositive
Theory rArr Phenomenon hArr notPhenomenonrArr notTheory
Additional tightening by Popper
Unexpected Phenomenon rarr More credence owed to the theoryeg Gravitational lensing rarr General theory of relativity
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Popperian Method
Logic behind the Popperian Method Contrapositive
Theory rArr Phenomenon hArr notPhenomenonrArr notTheory
Additional tightening by Popper
Unexpected Phenomenon rarr More credence owed to the theory
eg Gravitational lensing rarr General theory of relativity
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Additional Requirements in a Science of EC
Weak assumptions about distribution of fitness
This is just Occamrsquos Razor
Upfront proof of concept
Avoid another rdquoRoyal Roads momentrdquo
(Nice to have) Identification of a core computational efficiency
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Additional Requirements in a Science of EC
Weak assumptions about distribution of fitness
This is just Occamrsquos Razor
Upfront proof of concept
Avoid another rdquoRoyal Roads momentrdquo
(Nice to have) Identification of a core computational efficiency
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Additional Requirements in a Science of EC
Weak assumptions about distribution of fitness
This is just Occamrsquos Razor
Upfront proof of concept
Avoid another rdquoRoyal Roads momentrdquo
(Nice to have) Identification of a core computational efficiency
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Hypothesis
GAs with UX perform optimization by efficientlyimplementing a global search heuristic called hyperclimbing
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Outline
1 Highlight Symmetry of UX
2 Describe Hyperclimbing Heuristic
3 Provide proof-of-concept
Show a GA implementing hyperclimbing efficiently
4 Make a prediction and validate it on
MAX-3SATSherrington Kirkpatrick Spin Glasses problem
5 Outline future work
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0
1 0 0 1 1 0 0 1 1 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0X1 X2 X3 X4 X5 X6 X7 X8 X9 X`1 0 0 1 1 0 0 1 1 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0uarr darr darr uarr uarr darr darr darr uarr uarr1 0 0 1 1 0 0 1 1 1
0 0 0 0 1 0 0 1 1 0
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0uarr darr darr uarr uarr darr darr darr uarr uarr1 0 0 1 1 0 0 1 1 1
0 0 0 0 1 0 0 1 1 0Y1 Y2 Y3 Y4 Y5 Y6 Y7 Y8 Y9 Y`
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0uarr darr darr uarr uarr darr darr darr uarr uarr1 0 0 1 1 0 0 1 1 1
0 0 0 0 1 0 0 1 1 0dArr dArr dArr z dArr dArr dArr z dArr dArr0 0 0 1 1 0 0 0 1 0
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
X1 X2 X3 X4 X5 X6 X7 X8 X9 X`
Y1 Y2 Y3 Y4 Y5 Y6 Y7 Y8 Y9 Y`
UX X1 X` are independent
Absence of positional bias (Eshelman et al 1989)
Attributes can be arbitrarily permuted
Suppose Y1 Y` independent and independent of `
If locus i immaterial to fitness can be spliced out
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
The Hyperclimbing Heuristic
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing A Stochastic Search Heuristic
The Setting
0 1` frarr R
f may be stochastic
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing A Stochastic Search Heuristic
The Setting
0 1` frarr R
f may be stochastic
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
The Effect of a Schema Partition
Effect of JIK Variance of average fitness of schemata in JIK
If I sub I prime then Effect(I) le Effect(I prime)
JIK
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
The Effect of a Schema Partition
Effect of JIK Variance of average fitness of schemata in JIK
If I sub I prime then Effect(I) le Effect(I prime)
JIK JI primeK
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption is Weak
Staggered coarse conditional effects assumption is weakWeaker than fitness distribution assumption in the BBH
BBH assumes unstaggered coarse unconditional effects
Weaker assumptions are more likely to be satisfied in practice
Good bc we aspire to explain optimization in all GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption is Weak
Staggered coarse conditional effects assumption is weakWeaker than fitness distribution assumption in the BBH
BBH assumes unstaggered coarse unconditional effects
Weaker assumptions are more likely to be satisfied in practice
Good bc we aspire to explain optimization in all GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Proof Of Concept
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
fitness(x) simN (+025 1) if xi1 oplus xi2 oplus xi3 oplus xi4 = 1N (minus025 1) otherwise
0 Fitness
Pro
babi
lity
minus025 +025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
darr darr darr darr10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
uarr uarr uarr uarrEffective Loci
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Naive approach
Visit loci in groups of four
Check for differentiation inthe multivariate marginalfitness values
Time complexity is Ω(`4)
xi1 xi2 xi3 xi4
ExpectedMarginalFitness
0 0 0 0 minus0250 0 0 1 +0250 0 1 0 +0250 0 1 1 minus0250 1 0 0 +0250 1 0 1 minus0250 1 1 0 minus0250 1 1 1 +0251 0 0 0 +0251 0 0 1 minus0251 0 1 0 minus0251 0 1 1 +0251 1 0 0 minus0251 1 0 1 +0251 1 1 0 +0251 1 1 1 minus025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Visiting loci in groups ofthree or less wonrsquot work
xj
ExpectedMarginalFitness
0 001 00
xj1 xj2
ExpectedMarginalFitness
0 0 000 1 001 0 001 1 00
xj1 xj2 xj3
ExpectedMarginalFitness
0 0 0 000 0 1 000 1 0 000 1 1 001 0 0 001 0 1 001 1 0 001 1 1 00
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=200
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity200AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Symmetry Analytic Conclusion
Expected generations for the red dots to diverge constant wrt
Location of the red dots
Number of blue dots
Dynamics of a blue dot invariant to
Location of the red dots
Number of blue dots
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=1000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity1000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=10000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity10000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
Staircase function descriptor (m n δ σ ` LV )
L =
3 87 middot middot middot 93
42 58 middot middot middot 72
67 73 middot middot middot 81
︸ ︷︷ ︸
n
m V =
1 0 middot middot middot 11 0 middot middot middot 0
0 1 middot middot middot 1
︸ ︷︷ ︸
n
m
Staircase function is a stochastic function f 0 1` rarr R
0 δ 2δ (mminus1)δ mδ
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
δ = 02 σ = 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredInOrder__ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
trials=100 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
35 25 37 4631 5 32 2021 33 50 4240 27 30 312 14 48 2
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
B
Watch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
0 0 1 01 1 0 01 1 0 11 1 0 10 0 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=randommpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Prediction amp Validation
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
crosstype=2Mut=003Animmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
ctype=2StaircaseClampingAnimmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60000
100
200
300
400
500
600
700
800
900
Generation
Un
mu
tate
d lo
ci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
100
200
300
400
500
600
700
800
900
1000
Generation
Unm
uta
ted loci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Conclusion
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 4
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Crossover
Uniform Crossover first used by Ackley (1987)
Studied and popularized by Syswerda (1989) and Spears amp DeJong (1990 1991)
Numerous accounts of optimization in GAs with UX
In practice frequently outperforms XO with tight linkage(Fogel 2006)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Crossover
Uniform Crossover first used by Ackley (1987)
Studied and popularized by Syswerda (1989) and Spears amp DeJong (1990 1991)
Numerous accounts of optimization in GAs with UX
In practice frequently outperforms XO with tight linkage(Fogel 2006)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Optimization in GAs with UX Unexplained
Cannot be explained within the rubric of the BBH
No viable alternative has been proposed
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Optimization in GAs with UX Unexplained
Hyperclimbing Hypothesis
A scientific explanation for optimization in GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Does ldquoScientificrdquo Mean
Logical Positivism (Proof or Bust)
Scientific truth is absolute
Emphasis on Verifiability (ie mathematical proof)
The Popperian method
Scientific truth is provisional
Emphasis on Falsifiability (testable predictions)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Does ldquoScientificrdquo Mean
Logical Positivism (Proof or Bust)
Scientific truth is absolute
Emphasis on Verifiability (ie mathematical proof)
The Popperian method
Scientific truth is provisional
Emphasis on Falsifiability (testable predictions)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Popperian Method
Logic behind the Popperian Method Contrapositive
Theory rArr Phenomenon hArr notPhenomenonrArr notTheory
Additional tightening by Popper
Unexpected Phenomenon rarr More credence owed to the theoryeg Gravitational lensing rarr General theory of relativity
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Popperian Method
Logic behind the Popperian Method Contrapositive
Theory rArr Phenomenon hArr notPhenomenonrArr notTheory
Additional tightening by Popper
Unexpected Phenomenon rarr More credence owed to the theory
eg Gravitational lensing rarr General theory of relativity
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Additional Requirements in a Science of EC
Weak assumptions about distribution of fitness
This is just Occamrsquos Razor
Upfront proof of concept
Avoid another rdquoRoyal Roads momentrdquo
(Nice to have) Identification of a core computational efficiency
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Additional Requirements in a Science of EC
Weak assumptions about distribution of fitness
This is just Occamrsquos Razor
Upfront proof of concept
Avoid another rdquoRoyal Roads momentrdquo
(Nice to have) Identification of a core computational efficiency
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Additional Requirements in a Science of EC
Weak assumptions about distribution of fitness
This is just Occamrsquos Razor
Upfront proof of concept
Avoid another rdquoRoyal Roads momentrdquo
(Nice to have) Identification of a core computational efficiency
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Hypothesis
GAs with UX perform optimization by efficientlyimplementing a global search heuristic called hyperclimbing
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Outline
1 Highlight Symmetry of UX
2 Describe Hyperclimbing Heuristic
3 Provide proof-of-concept
Show a GA implementing hyperclimbing efficiently
4 Make a prediction and validate it on
MAX-3SATSherrington Kirkpatrick Spin Glasses problem
5 Outline future work
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0
1 0 0 1 1 0 0 1 1 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0X1 X2 X3 X4 X5 X6 X7 X8 X9 X`1 0 0 1 1 0 0 1 1 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0uarr darr darr uarr uarr darr darr darr uarr uarr1 0 0 1 1 0 0 1 1 1
0 0 0 0 1 0 0 1 1 0
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0uarr darr darr uarr uarr darr darr darr uarr uarr1 0 0 1 1 0 0 1 1 1
0 0 0 0 1 0 0 1 1 0Y1 Y2 Y3 Y4 Y5 Y6 Y7 Y8 Y9 Y`
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0uarr darr darr uarr uarr darr darr darr uarr uarr1 0 0 1 1 0 0 1 1 1
0 0 0 0 1 0 0 1 1 0dArr dArr dArr z dArr dArr dArr z dArr dArr0 0 0 1 1 0 0 0 1 0
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
X1 X2 X3 X4 X5 X6 X7 X8 X9 X`
Y1 Y2 Y3 Y4 Y5 Y6 Y7 Y8 Y9 Y`
UX X1 X` are independent
Absence of positional bias (Eshelman et al 1989)
Attributes can be arbitrarily permuted
Suppose Y1 Y` independent and independent of `
If locus i immaterial to fitness can be spliced out
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
The Hyperclimbing Heuristic
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing A Stochastic Search Heuristic
The Setting
0 1` frarr R
f may be stochastic
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing A Stochastic Search Heuristic
The Setting
0 1` frarr R
f may be stochastic
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
The Effect of a Schema Partition
Effect of JIK Variance of average fitness of schemata in JIK
If I sub I prime then Effect(I) le Effect(I prime)
JIK
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
The Effect of a Schema Partition
Effect of JIK Variance of average fitness of schemata in JIK
If I sub I prime then Effect(I) le Effect(I prime)
JIK JI primeK
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption is Weak
Staggered coarse conditional effects assumption is weakWeaker than fitness distribution assumption in the BBH
BBH assumes unstaggered coarse unconditional effects
Weaker assumptions are more likely to be satisfied in practice
Good bc we aspire to explain optimization in all GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption is Weak
Staggered coarse conditional effects assumption is weakWeaker than fitness distribution assumption in the BBH
BBH assumes unstaggered coarse unconditional effects
Weaker assumptions are more likely to be satisfied in practice
Good bc we aspire to explain optimization in all GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Proof Of Concept
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
fitness(x) simN (+025 1) if xi1 oplus xi2 oplus xi3 oplus xi4 = 1N (minus025 1) otherwise
0 Fitness
Pro
babi
lity
minus025 +025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
darr darr darr darr10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
uarr uarr uarr uarrEffective Loci
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Naive approach
Visit loci in groups of four
Check for differentiation inthe multivariate marginalfitness values
Time complexity is Ω(`4)
xi1 xi2 xi3 xi4
ExpectedMarginalFitness
0 0 0 0 minus0250 0 0 1 +0250 0 1 0 +0250 0 1 1 minus0250 1 0 0 +0250 1 0 1 minus0250 1 1 0 minus0250 1 1 1 +0251 0 0 0 +0251 0 0 1 minus0251 0 1 0 minus0251 0 1 1 +0251 1 0 0 minus0251 1 0 1 +0251 1 1 0 +0251 1 1 1 minus025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Visiting loci in groups ofthree or less wonrsquot work
xj
ExpectedMarginalFitness
0 001 00
xj1 xj2
ExpectedMarginalFitness
0 0 000 1 001 0 001 1 00
xj1 xj2 xj3
ExpectedMarginalFitness
0 0 0 000 0 1 000 1 0 000 1 1 001 0 0 001 0 1 001 1 0 001 1 1 00
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=200
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity200AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Symmetry Analytic Conclusion
Expected generations for the red dots to diverge constant wrt
Location of the red dots
Number of blue dots
Dynamics of a blue dot invariant to
Location of the red dots
Number of blue dots
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=1000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity1000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=10000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity10000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
Staircase function descriptor (m n δ σ ` LV )
L =
3 87 middot middot middot 93
42 58 middot middot middot 72
67 73 middot middot middot 81
︸ ︷︷ ︸
n
m V =
1 0 middot middot middot 11 0 middot middot middot 0
0 1 middot middot middot 1
︸ ︷︷ ︸
n
m
Staircase function is a stochastic function f 0 1` rarr R
0 δ 2δ (mminus1)δ mδ
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
δ = 02 σ = 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredInOrder__ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
trials=100 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
35 25 37 4631 5 32 2021 33 50 4240 27 30 312 14 48 2
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
B
Watch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
0 0 1 01 1 0 01 1 0 11 1 0 10 0 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=randommpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Prediction amp Validation
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
crosstype=2Mut=003Animmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
ctype=2StaircaseClampingAnimmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60000
100
200
300
400
500
600
700
800
900
Generation
Un
mu
tate
d lo
ci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
100
200
300
400
500
600
700
800
900
1000
Generation
Unm
uta
ted loci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Conclusion
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 5
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Crossover
Uniform Crossover first used by Ackley (1987)
Studied and popularized by Syswerda (1989) and Spears amp DeJong (1990 1991)
Numerous accounts of optimization in GAs with UX
In practice frequently outperforms XO with tight linkage(Fogel 2006)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Optimization in GAs with UX Unexplained
Cannot be explained within the rubric of the BBH
No viable alternative has been proposed
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Optimization in GAs with UX Unexplained
Hyperclimbing Hypothesis
A scientific explanation for optimization in GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Does ldquoScientificrdquo Mean
Logical Positivism (Proof or Bust)
Scientific truth is absolute
Emphasis on Verifiability (ie mathematical proof)
The Popperian method
Scientific truth is provisional
Emphasis on Falsifiability (testable predictions)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Does ldquoScientificrdquo Mean
Logical Positivism (Proof or Bust)
Scientific truth is absolute
Emphasis on Verifiability (ie mathematical proof)
The Popperian method
Scientific truth is provisional
Emphasis on Falsifiability (testable predictions)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Popperian Method
Logic behind the Popperian Method Contrapositive
Theory rArr Phenomenon hArr notPhenomenonrArr notTheory
Additional tightening by Popper
Unexpected Phenomenon rarr More credence owed to the theoryeg Gravitational lensing rarr General theory of relativity
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Popperian Method
Logic behind the Popperian Method Contrapositive
Theory rArr Phenomenon hArr notPhenomenonrArr notTheory
Additional tightening by Popper
Unexpected Phenomenon rarr More credence owed to the theory
eg Gravitational lensing rarr General theory of relativity
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Additional Requirements in a Science of EC
Weak assumptions about distribution of fitness
This is just Occamrsquos Razor
Upfront proof of concept
Avoid another rdquoRoyal Roads momentrdquo
(Nice to have) Identification of a core computational efficiency
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Additional Requirements in a Science of EC
Weak assumptions about distribution of fitness
This is just Occamrsquos Razor
Upfront proof of concept
Avoid another rdquoRoyal Roads momentrdquo
(Nice to have) Identification of a core computational efficiency
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Additional Requirements in a Science of EC
Weak assumptions about distribution of fitness
This is just Occamrsquos Razor
Upfront proof of concept
Avoid another rdquoRoyal Roads momentrdquo
(Nice to have) Identification of a core computational efficiency
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Hypothesis
GAs with UX perform optimization by efficientlyimplementing a global search heuristic called hyperclimbing
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Outline
1 Highlight Symmetry of UX
2 Describe Hyperclimbing Heuristic
3 Provide proof-of-concept
Show a GA implementing hyperclimbing efficiently
4 Make a prediction and validate it on
MAX-3SATSherrington Kirkpatrick Spin Glasses problem
5 Outline future work
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0
1 0 0 1 1 0 0 1 1 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0X1 X2 X3 X4 X5 X6 X7 X8 X9 X`1 0 0 1 1 0 0 1 1 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0uarr darr darr uarr uarr darr darr darr uarr uarr1 0 0 1 1 0 0 1 1 1
0 0 0 0 1 0 0 1 1 0
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0uarr darr darr uarr uarr darr darr darr uarr uarr1 0 0 1 1 0 0 1 1 1
0 0 0 0 1 0 0 1 1 0Y1 Y2 Y3 Y4 Y5 Y6 Y7 Y8 Y9 Y`
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0uarr darr darr uarr uarr darr darr darr uarr uarr1 0 0 1 1 0 0 1 1 1
0 0 0 0 1 0 0 1 1 0dArr dArr dArr z dArr dArr dArr z dArr dArr0 0 0 1 1 0 0 0 1 0
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
X1 X2 X3 X4 X5 X6 X7 X8 X9 X`
Y1 Y2 Y3 Y4 Y5 Y6 Y7 Y8 Y9 Y`
UX X1 X` are independent
Absence of positional bias (Eshelman et al 1989)
Attributes can be arbitrarily permuted
Suppose Y1 Y` independent and independent of `
If locus i immaterial to fitness can be spliced out
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
The Hyperclimbing Heuristic
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing A Stochastic Search Heuristic
The Setting
0 1` frarr R
f may be stochastic
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing A Stochastic Search Heuristic
The Setting
0 1` frarr R
f may be stochastic
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
The Effect of a Schema Partition
Effect of JIK Variance of average fitness of schemata in JIK
If I sub I prime then Effect(I) le Effect(I prime)
JIK
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
The Effect of a Schema Partition
Effect of JIK Variance of average fitness of schemata in JIK
If I sub I prime then Effect(I) le Effect(I prime)
JIK JI primeK
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption is Weak
Staggered coarse conditional effects assumption is weakWeaker than fitness distribution assumption in the BBH
BBH assumes unstaggered coarse unconditional effects
Weaker assumptions are more likely to be satisfied in practice
Good bc we aspire to explain optimization in all GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption is Weak
Staggered coarse conditional effects assumption is weakWeaker than fitness distribution assumption in the BBH
BBH assumes unstaggered coarse unconditional effects
Weaker assumptions are more likely to be satisfied in practice
Good bc we aspire to explain optimization in all GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Proof Of Concept
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
fitness(x) simN (+025 1) if xi1 oplus xi2 oplus xi3 oplus xi4 = 1N (minus025 1) otherwise
0 Fitness
Pro
babi
lity
minus025 +025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
darr darr darr darr10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
uarr uarr uarr uarrEffective Loci
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Naive approach
Visit loci in groups of four
Check for differentiation inthe multivariate marginalfitness values
Time complexity is Ω(`4)
xi1 xi2 xi3 xi4
ExpectedMarginalFitness
0 0 0 0 minus0250 0 0 1 +0250 0 1 0 +0250 0 1 1 minus0250 1 0 0 +0250 1 0 1 minus0250 1 1 0 minus0250 1 1 1 +0251 0 0 0 +0251 0 0 1 minus0251 0 1 0 minus0251 0 1 1 +0251 1 0 0 minus0251 1 0 1 +0251 1 1 0 +0251 1 1 1 minus025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Visiting loci in groups ofthree or less wonrsquot work
xj
ExpectedMarginalFitness
0 001 00
xj1 xj2
ExpectedMarginalFitness
0 0 000 1 001 0 001 1 00
xj1 xj2 xj3
ExpectedMarginalFitness
0 0 0 000 0 1 000 1 0 000 1 1 001 0 0 001 0 1 001 1 0 001 1 1 00
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=200
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity200AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Symmetry Analytic Conclusion
Expected generations for the red dots to diverge constant wrt
Location of the red dots
Number of blue dots
Dynamics of a blue dot invariant to
Location of the red dots
Number of blue dots
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=1000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity1000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=10000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity10000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
Staircase function descriptor (m n δ σ ` LV )
L =
3 87 middot middot middot 93
42 58 middot middot middot 72
67 73 middot middot middot 81
︸ ︷︷ ︸
n
m V =
1 0 middot middot middot 11 0 middot middot middot 0
0 1 middot middot middot 1
︸ ︷︷ ︸
n
m
Staircase function is a stochastic function f 0 1` rarr R
0 δ 2δ (mminus1)δ mδ
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
δ = 02 σ = 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredInOrder__ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
trials=100 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
35 25 37 4631 5 32 2021 33 50 4240 27 30 312 14 48 2
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
B
Watch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
0 0 1 01 1 0 01 1 0 11 1 0 10 0 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=randommpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Prediction amp Validation
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
crosstype=2Mut=003Animmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
ctype=2StaircaseClampingAnimmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60000
100
200
300
400
500
600
700
800
900
Generation
Un
mu
tate
d lo
ci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
100
200
300
400
500
600
700
800
900
1000
Generation
Unm
uta
ted loci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Conclusion
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 6
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Optimization in GAs with UX Unexplained
Cannot be explained within the rubric of the BBH
No viable alternative has been proposed
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Optimization in GAs with UX Unexplained
Hyperclimbing Hypothesis
A scientific explanation for optimization in GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Does ldquoScientificrdquo Mean
Logical Positivism (Proof or Bust)
Scientific truth is absolute
Emphasis on Verifiability (ie mathematical proof)
The Popperian method
Scientific truth is provisional
Emphasis on Falsifiability (testable predictions)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Does ldquoScientificrdquo Mean
Logical Positivism (Proof or Bust)
Scientific truth is absolute
Emphasis on Verifiability (ie mathematical proof)
The Popperian method
Scientific truth is provisional
Emphasis on Falsifiability (testable predictions)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Popperian Method
Logic behind the Popperian Method Contrapositive
Theory rArr Phenomenon hArr notPhenomenonrArr notTheory
Additional tightening by Popper
Unexpected Phenomenon rarr More credence owed to the theoryeg Gravitational lensing rarr General theory of relativity
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Popperian Method
Logic behind the Popperian Method Contrapositive
Theory rArr Phenomenon hArr notPhenomenonrArr notTheory
Additional tightening by Popper
Unexpected Phenomenon rarr More credence owed to the theory
eg Gravitational lensing rarr General theory of relativity
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Additional Requirements in a Science of EC
Weak assumptions about distribution of fitness
This is just Occamrsquos Razor
Upfront proof of concept
Avoid another rdquoRoyal Roads momentrdquo
(Nice to have) Identification of a core computational efficiency
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Additional Requirements in a Science of EC
Weak assumptions about distribution of fitness
This is just Occamrsquos Razor
Upfront proof of concept
Avoid another rdquoRoyal Roads momentrdquo
(Nice to have) Identification of a core computational efficiency
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Additional Requirements in a Science of EC
Weak assumptions about distribution of fitness
This is just Occamrsquos Razor
Upfront proof of concept
Avoid another rdquoRoyal Roads momentrdquo
(Nice to have) Identification of a core computational efficiency
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Hypothesis
GAs with UX perform optimization by efficientlyimplementing a global search heuristic called hyperclimbing
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Outline
1 Highlight Symmetry of UX
2 Describe Hyperclimbing Heuristic
3 Provide proof-of-concept
Show a GA implementing hyperclimbing efficiently
4 Make a prediction and validate it on
MAX-3SATSherrington Kirkpatrick Spin Glasses problem
5 Outline future work
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0
1 0 0 1 1 0 0 1 1 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0X1 X2 X3 X4 X5 X6 X7 X8 X9 X`1 0 0 1 1 0 0 1 1 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0uarr darr darr uarr uarr darr darr darr uarr uarr1 0 0 1 1 0 0 1 1 1
0 0 0 0 1 0 0 1 1 0
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0uarr darr darr uarr uarr darr darr darr uarr uarr1 0 0 1 1 0 0 1 1 1
0 0 0 0 1 0 0 1 1 0Y1 Y2 Y3 Y4 Y5 Y6 Y7 Y8 Y9 Y`
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0uarr darr darr uarr uarr darr darr darr uarr uarr1 0 0 1 1 0 0 1 1 1
0 0 0 0 1 0 0 1 1 0dArr dArr dArr z dArr dArr dArr z dArr dArr0 0 0 1 1 0 0 0 1 0
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
X1 X2 X3 X4 X5 X6 X7 X8 X9 X`
Y1 Y2 Y3 Y4 Y5 Y6 Y7 Y8 Y9 Y`
UX X1 X` are independent
Absence of positional bias (Eshelman et al 1989)
Attributes can be arbitrarily permuted
Suppose Y1 Y` independent and independent of `
If locus i immaterial to fitness can be spliced out
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
The Hyperclimbing Heuristic
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing A Stochastic Search Heuristic
The Setting
0 1` frarr R
f may be stochastic
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing A Stochastic Search Heuristic
The Setting
0 1` frarr R
f may be stochastic
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
The Effect of a Schema Partition
Effect of JIK Variance of average fitness of schemata in JIK
If I sub I prime then Effect(I) le Effect(I prime)
JIK
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
The Effect of a Schema Partition
Effect of JIK Variance of average fitness of schemata in JIK
If I sub I prime then Effect(I) le Effect(I prime)
JIK JI primeK
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption is Weak
Staggered coarse conditional effects assumption is weakWeaker than fitness distribution assumption in the BBH
BBH assumes unstaggered coarse unconditional effects
Weaker assumptions are more likely to be satisfied in practice
Good bc we aspire to explain optimization in all GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption is Weak
Staggered coarse conditional effects assumption is weakWeaker than fitness distribution assumption in the BBH
BBH assumes unstaggered coarse unconditional effects
Weaker assumptions are more likely to be satisfied in practice
Good bc we aspire to explain optimization in all GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Proof Of Concept
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
fitness(x) simN (+025 1) if xi1 oplus xi2 oplus xi3 oplus xi4 = 1N (minus025 1) otherwise
0 Fitness
Pro
babi
lity
minus025 +025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
darr darr darr darr10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
uarr uarr uarr uarrEffective Loci
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Naive approach
Visit loci in groups of four
Check for differentiation inthe multivariate marginalfitness values
Time complexity is Ω(`4)
xi1 xi2 xi3 xi4
ExpectedMarginalFitness
0 0 0 0 minus0250 0 0 1 +0250 0 1 0 +0250 0 1 1 minus0250 1 0 0 +0250 1 0 1 minus0250 1 1 0 minus0250 1 1 1 +0251 0 0 0 +0251 0 0 1 minus0251 0 1 0 minus0251 0 1 1 +0251 1 0 0 minus0251 1 0 1 +0251 1 1 0 +0251 1 1 1 minus025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Visiting loci in groups ofthree or less wonrsquot work
xj
ExpectedMarginalFitness
0 001 00
xj1 xj2
ExpectedMarginalFitness
0 0 000 1 001 0 001 1 00
xj1 xj2 xj3
ExpectedMarginalFitness
0 0 0 000 0 1 000 1 0 000 1 1 001 0 0 001 0 1 001 1 0 001 1 1 00
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=200
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity200AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Symmetry Analytic Conclusion
Expected generations for the red dots to diverge constant wrt
Location of the red dots
Number of blue dots
Dynamics of a blue dot invariant to
Location of the red dots
Number of blue dots
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=1000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity1000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=10000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity10000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
Staircase function descriptor (m n δ σ ` LV )
L =
3 87 middot middot middot 93
42 58 middot middot middot 72
67 73 middot middot middot 81
︸ ︷︷ ︸
n
m V =
1 0 middot middot middot 11 0 middot middot middot 0
0 1 middot middot middot 1
︸ ︷︷ ︸
n
m
Staircase function is a stochastic function f 0 1` rarr R
0 δ 2δ (mminus1)δ mδ
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
δ = 02 σ = 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredInOrder__ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
trials=100 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
35 25 37 4631 5 32 2021 33 50 4240 27 30 312 14 48 2
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
B
Watch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
0 0 1 01 1 0 01 1 0 11 1 0 10 0 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=randommpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Prediction amp Validation
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
crosstype=2Mut=003Animmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
ctype=2StaircaseClampingAnimmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60000
100
200
300
400
500
600
700
800
900
Generation
Un
mu
tate
d lo
ci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
100
200
300
400
500
600
700
800
900
1000
Generation
Unm
uta
ted loci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Conclusion
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 7
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Optimization in GAs with UX Unexplained
Hyperclimbing Hypothesis
A scientific explanation for optimization in GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Does ldquoScientificrdquo Mean
Logical Positivism (Proof or Bust)
Scientific truth is absolute
Emphasis on Verifiability (ie mathematical proof)
The Popperian method
Scientific truth is provisional
Emphasis on Falsifiability (testable predictions)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Does ldquoScientificrdquo Mean
Logical Positivism (Proof or Bust)
Scientific truth is absolute
Emphasis on Verifiability (ie mathematical proof)
The Popperian method
Scientific truth is provisional
Emphasis on Falsifiability (testable predictions)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Popperian Method
Logic behind the Popperian Method Contrapositive
Theory rArr Phenomenon hArr notPhenomenonrArr notTheory
Additional tightening by Popper
Unexpected Phenomenon rarr More credence owed to the theoryeg Gravitational lensing rarr General theory of relativity
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Popperian Method
Logic behind the Popperian Method Contrapositive
Theory rArr Phenomenon hArr notPhenomenonrArr notTheory
Additional tightening by Popper
Unexpected Phenomenon rarr More credence owed to the theory
eg Gravitational lensing rarr General theory of relativity
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Additional Requirements in a Science of EC
Weak assumptions about distribution of fitness
This is just Occamrsquos Razor
Upfront proof of concept
Avoid another rdquoRoyal Roads momentrdquo
(Nice to have) Identification of a core computational efficiency
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Additional Requirements in a Science of EC
Weak assumptions about distribution of fitness
This is just Occamrsquos Razor
Upfront proof of concept
Avoid another rdquoRoyal Roads momentrdquo
(Nice to have) Identification of a core computational efficiency
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Additional Requirements in a Science of EC
Weak assumptions about distribution of fitness
This is just Occamrsquos Razor
Upfront proof of concept
Avoid another rdquoRoyal Roads momentrdquo
(Nice to have) Identification of a core computational efficiency
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Hypothesis
GAs with UX perform optimization by efficientlyimplementing a global search heuristic called hyperclimbing
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Outline
1 Highlight Symmetry of UX
2 Describe Hyperclimbing Heuristic
3 Provide proof-of-concept
Show a GA implementing hyperclimbing efficiently
4 Make a prediction and validate it on
MAX-3SATSherrington Kirkpatrick Spin Glasses problem
5 Outline future work
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0
1 0 0 1 1 0 0 1 1 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0X1 X2 X3 X4 X5 X6 X7 X8 X9 X`1 0 0 1 1 0 0 1 1 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0uarr darr darr uarr uarr darr darr darr uarr uarr1 0 0 1 1 0 0 1 1 1
0 0 0 0 1 0 0 1 1 0
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0uarr darr darr uarr uarr darr darr darr uarr uarr1 0 0 1 1 0 0 1 1 1
0 0 0 0 1 0 0 1 1 0Y1 Y2 Y3 Y4 Y5 Y6 Y7 Y8 Y9 Y`
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0uarr darr darr uarr uarr darr darr darr uarr uarr1 0 0 1 1 0 0 1 1 1
0 0 0 0 1 0 0 1 1 0dArr dArr dArr z dArr dArr dArr z dArr dArr0 0 0 1 1 0 0 0 1 0
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
X1 X2 X3 X4 X5 X6 X7 X8 X9 X`
Y1 Y2 Y3 Y4 Y5 Y6 Y7 Y8 Y9 Y`
UX X1 X` are independent
Absence of positional bias (Eshelman et al 1989)
Attributes can be arbitrarily permuted
Suppose Y1 Y` independent and independent of `
If locus i immaterial to fitness can be spliced out
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
The Hyperclimbing Heuristic
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing A Stochastic Search Heuristic
The Setting
0 1` frarr R
f may be stochastic
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing A Stochastic Search Heuristic
The Setting
0 1` frarr R
f may be stochastic
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
The Effect of a Schema Partition
Effect of JIK Variance of average fitness of schemata in JIK
If I sub I prime then Effect(I) le Effect(I prime)
JIK
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
The Effect of a Schema Partition
Effect of JIK Variance of average fitness of schemata in JIK
If I sub I prime then Effect(I) le Effect(I prime)
JIK JI primeK
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption is Weak
Staggered coarse conditional effects assumption is weakWeaker than fitness distribution assumption in the BBH
BBH assumes unstaggered coarse unconditional effects
Weaker assumptions are more likely to be satisfied in practice
Good bc we aspire to explain optimization in all GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption is Weak
Staggered coarse conditional effects assumption is weakWeaker than fitness distribution assumption in the BBH
BBH assumes unstaggered coarse unconditional effects
Weaker assumptions are more likely to be satisfied in practice
Good bc we aspire to explain optimization in all GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Proof Of Concept
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
fitness(x) simN (+025 1) if xi1 oplus xi2 oplus xi3 oplus xi4 = 1N (minus025 1) otherwise
0 Fitness
Pro
babi
lity
minus025 +025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
darr darr darr darr10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
uarr uarr uarr uarrEffective Loci
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Naive approach
Visit loci in groups of four
Check for differentiation inthe multivariate marginalfitness values
Time complexity is Ω(`4)
xi1 xi2 xi3 xi4
ExpectedMarginalFitness
0 0 0 0 minus0250 0 0 1 +0250 0 1 0 +0250 0 1 1 minus0250 1 0 0 +0250 1 0 1 minus0250 1 1 0 minus0250 1 1 1 +0251 0 0 0 +0251 0 0 1 minus0251 0 1 0 minus0251 0 1 1 +0251 1 0 0 minus0251 1 0 1 +0251 1 1 0 +0251 1 1 1 minus025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Visiting loci in groups ofthree or less wonrsquot work
xj
ExpectedMarginalFitness
0 001 00
xj1 xj2
ExpectedMarginalFitness
0 0 000 1 001 0 001 1 00
xj1 xj2 xj3
ExpectedMarginalFitness
0 0 0 000 0 1 000 1 0 000 1 1 001 0 0 001 0 1 001 1 0 001 1 1 00
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=200
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity200AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Symmetry Analytic Conclusion
Expected generations for the red dots to diverge constant wrt
Location of the red dots
Number of blue dots
Dynamics of a blue dot invariant to
Location of the red dots
Number of blue dots
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=1000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity1000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=10000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity10000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
Staircase function descriptor (m n δ σ ` LV )
L =
3 87 middot middot middot 93
42 58 middot middot middot 72
67 73 middot middot middot 81
︸ ︷︷ ︸
n
m V =
1 0 middot middot middot 11 0 middot middot middot 0
0 1 middot middot middot 1
︸ ︷︷ ︸
n
m
Staircase function is a stochastic function f 0 1` rarr R
0 δ 2δ (mminus1)δ mδ
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
δ = 02 σ = 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredInOrder__ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
trials=100 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
35 25 37 4631 5 32 2021 33 50 4240 27 30 312 14 48 2
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
B
Watch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
0 0 1 01 1 0 01 1 0 11 1 0 10 0 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=randommpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Prediction amp Validation
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
crosstype=2Mut=003Animmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
ctype=2StaircaseClampingAnimmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60000
100
200
300
400
500
600
700
800
900
Generation
Un
mu
tate
d lo
ci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
100
200
300
400
500
600
700
800
900
1000
Generation
Unm
uta
ted loci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Conclusion
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 8
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Does ldquoScientificrdquo Mean
Logical Positivism (Proof or Bust)
Scientific truth is absolute
Emphasis on Verifiability (ie mathematical proof)
The Popperian method
Scientific truth is provisional
Emphasis on Falsifiability (testable predictions)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Does ldquoScientificrdquo Mean
Logical Positivism (Proof or Bust)
Scientific truth is absolute
Emphasis on Verifiability (ie mathematical proof)
The Popperian method
Scientific truth is provisional
Emphasis on Falsifiability (testable predictions)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Popperian Method
Logic behind the Popperian Method Contrapositive
Theory rArr Phenomenon hArr notPhenomenonrArr notTheory
Additional tightening by Popper
Unexpected Phenomenon rarr More credence owed to the theoryeg Gravitational lensing rarr General theory of relativity
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Popperian Method
Logic behind the Popperian Method Contrapositive
Theory rArr Phenomenon hArr notPhenomenonrArr notTheory
Additional tightening by Popper
Unexpected Phenomenon rarr More credence owed to the theory
eg Gravitational lensing rarr General theory of relativity
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Additional Requirements in a Science of EC
Weak assumptions about distribution of fitness
This is just Occamrsquos Razor
Upfront proof of concept
Avoid another rdquoRoyal Roads momentrdquo
(Nice to have) Identification of a core computational efficiency
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Additional Requirements in a Science of EC
Weak assumptions about distribution of fitness
This is just Occamrsquos Razor
Upfront proof of concept
Avoid another rdquoRoyal Roads momentrdquo
(Nice to have) Identification of a core computational efficiency
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Additional Requirements in a Science of EC
Weak assumptions about distribution of fitness
This is just Occamrsquos Razor
Upfront proof of concept
Avoid another rdquoRoyal Roads momentrdquo
(Nice to have) Identification of a core computational efficiency
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Hypothesis
GAs with UX perform optimization by efficientlyimplementing a global search heuristic called hyperclimbing
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Outline
1 Highlight Symmetry of UX
2 Describe Hyperclimbing Heuristic
3 Provide proof-of-concept
Show a GA implementing hyperclimbing efficiently
4 Make a prediction and validate it on
MAX-3SATSherrington Kirkpatrick Spin Glasses problem
5 Outline future work
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0
1 0 0 1 1 0 0 1 1 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0X1 X2 X3 X4 X5 X6 X7 X8 X9 X`1 0 0 1 1 0 0 1 1 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0uarr darr darr uarr uarr darr darr darr uarr uarr1 0 0 1 1 0 0 1 1 1
0 0 0 0 1 0 0 1 1 0
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0uarr darr darr uarr uarr darr darr darr uarr uarr1 0 0 1 1 0 0 1 1 1
0 0 0 0 1 0 0 1 1 0Y1 Y2 Y3 Y4 Y5 Y6 Y7 Y8 Y9 Y`
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0uarr darr darr uarr uarr darr darr darr uarr uarr1 0 0 1 1 0 0 1 1 1
0 0 0 0 1 0 0 1 1 0dArr dArr dArr z dArr dArr dArr z dArr dArr0 0 0 1 1 0 0 0 1 0
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
X1 X2 X3 X4 X5 X6 X7 X8 X9 X`
Y1 Y2 Y3 Y4 Y5 Y6 Y7 Y8 Y9 Y`
UX X1 X` are independent
Absence of positional bias (Eshelman et al 1989)
Attributes can be arbitrarily permuted
Suppose Y1 Y` independent and independent of `
If locus i immaterial to fitness can be spliced out
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
The Hyperclimbing Heuristic
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing A Stochastic Search Heuristic
The Setting
0 1` frarr R
f may be stochastic
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing A Stochastic Search Heuristic
The Setting
0 1` frarr R
f may be stochastic
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
The Effect of a Schema Partition
Effect of JIK Variance of average fitness of schemata in JIK
If I sub I prime then Effect(I) le Effect(I prime)
JIK
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
The Effect of a Schema Partition
Effect of JIK Variance of average fitness of schemata in JIK
If I sub I prime then Effect(I) le Effect(I prime)
JIK JI primeK
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption is Weak
Staggered coarse conditional effects assumption is weakWeaker than fitness distribution assumption in the BBH
BBH assumes unstaggered coarse unconditional effects
Weaker assumptions are more likely to be satisfied in practice
Good bc we aspire to explain optimization in all GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption is Weak
Staggered coarse conditional effects assumption is weakWeaker than fitness distribution assumption in the BBH
BBH assumes unstaggered coarse unconditional effects
Weaker assumptions are more likely to be satisfied in practice
Good bc we aspire to explain optimization in all GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Proof Of Concept
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
fitness(x) simN (+025 1) if xi1 oplus xi2 oplus xi3 oplus xi4 = 1N (minus025 1) otherwise
0 Fitness
Pro
babi
lity
minus025 +025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
darr darr darr darr10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
uarr uarr uarr uarrEffective Loci
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Naive approach
Visit loci in groups of four
Check for differentiation inthe multivariate marginalfitness values
Time complexity is Ω(`4)
xi1 xi2 xi3 xi4
ExpectedMarginalFitness
0 0 0 0 minus0250 0 0 1 +0250 0 1 0 +0250 0 1 1 minus0250 1 0 0 +0250 1 0 1 minus0250 1 1 0 minus0250 1 1 1 +0251 0 0 0 +0251 0 0 1 minus0251 0 1 0 minus0251 0 1 1 +0251 1 0 0 minus0251 1 0 1 +0251 1 1 0 +0251 1 1 1 minus025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Visiting loci in groups ofthree or less wonrsquot work
xj
ExpectedMarginalFitness
0 001 00
xj1 xj2
ExpectedMarginalFitness
0 0 000 1 001 0 001 1 00
xj1 xj2 xj3
ExpectedMarginalFitness
0 0 0 000 0 1 000 1 0 000 1 1 001 0 0 001 0 1 001 1 0 001 1 1 00
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=200
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity200AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Symmetry Analytic Conclusion
Expected generations for the red dots to diverge constant wrt
Location of the red dots
Number of blue dots
Dynamics of a blue dot invariant to
Location of the red dots
Number of blue dots
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=1000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity1000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=10000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity10000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
Staircase function descriptor (m n δ σ ` LV )
L =
3 87 middot middot middot 93
42 58 middot middot middot 72
67 73 middot middot middot 81
︸ ︷︷ ︸
n
m V =
1 0 middot middot middot 11 0 middot middot middot 0
0 1 middot middot middot 1
︸ ︷︷ ︸
n
m
Staircase function is a stochastic function f 0 1` rarr R
0 δ 2δ (mminus1)δ mδ
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
δ = 02 σ = 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredInOrder__ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
trials=100 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
35 25 37 4631 5 32 2021 33 50 4240 27 30 312 14 48 2
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
B
Watch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
0 0 1 01 1 0 01 1 0 11 1 0 10 0 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=randommpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Prediction amp Validation
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
crosstype=2Mut=003Animmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
ctype=2StaircaseClampingAnimmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60000
100
200
300
400
500
600
700
800
900
Generation
Un
mu
tate
d lo
ci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
100
200
300
400
500
600
700
800
900
1000
Generation
Unm
uta
ted loci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Conclusion
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 9
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Does ldquoScientificrdquo Mean
Logical Positivism (Proof or Bust)
Scientific truth is absolute
Emphasis on Verifiability (ie mathematical proof)
The Popperian method
Scientific truth is provisional
Emphasis on Falsifiability (testable predictions)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Popperian Method
Logic behind the Popperian Method Contrapositive
Theory rArr Phenomenon hArr notPhenomenonrArr notTheory
Additional tightening by Popper
Unexpected Phenomenon rarr More credence owed to the theoryeg Gravitational lensing rarr General theory of relativity
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Popperian Method
Logic behind the Popperian Method Contrapositive
Theory rArr Phenomenon hArr notPhenomenonrArr notTheory
Additional tightening by Popper
Unexpected Phenomenon rarr More credence owed to the theory
eg Gravitational lensing rarr General theory of relativity
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Additional Requirements in a Science of EC
Weak assumptions about distribution of fitness
This is just Occamrsquos Razor
Upfront proof of concept
Avoid another rdquoRoyal Roads momentrdquo
(Nice to have) Identification of a core computational efficiency
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Additional Requirements in a Science of EC
Weak assumptions about distribution of fitness
This is just Occamrsquos Razor
Upfront proof of concept
Avoid another rdquoRoyal Roads momentrdquo
(Nice to have) Identification of a core computational efficiency
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Additional Requirements in a Science of EC
Weak assumptions about distribution of fitness
This is just Occamrsquos Razor
Upfront proof of concept
Avoid another rdquoRoyal Roads momentrdquo
(Nice to have) Identification of a core computational efficiency
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Hypothesis
GAs with UX perform optimization by efficientlyimplementing a global search heuristic called hyperclimbing
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Outline
1 Highlight Symmetry of UX
2 Describe Hyperclimbing Heuristic
3 Provide proof-of-concept
Show a GA implementing hyperclimbing efficiently
4 Make a prediction and validate it on
MAX-3SATSherrington Kirkpatrick Spin Glasses problem
5 Outline future work
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0
1 0 0 1 1 0 0 1 1 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0X1 X2 X3 X4 X5 X6 X7 X8 X9 X`1 0 0 1 1 0 0 1 1 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0uarr darr darr uarr uarr darr darr darr uarr uarr1 0 0 1 1 0 0 1 1 1
0 0 0 0 1 0 0 1 1 0
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0uarr darr darr uarr uarr darr darr darr uarr uarr1 0 0 1 1 0 0 1 1 1
0 0 0 0 1 0 0 1 1 0Y1 Y2 Y3 Y4 Y5 Y6 Y7 Y8 Y9 Y`
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0uarr darr darr uarr uarr darr darr darr uarr uarr1 0 0 1 1 0 0 1 1 1
0 0 0 0 1 0 0 1 1 0dArr dArr dArr z dArr dArr dArr z dArr dArr0 0 0 1 1 0 0 0 1 0
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
X1 X2 X3 X4 X5 X6 X7 X8 X9 X`
Y1 Y2 Y3 Y4 Y5 Y6 Y7 Y8 Y9 Y`
UX X1 X` are independent
Absence of positional bias (Eshelman et al 1989)
Attributes can be arbitrarily permuted
Suppose Y1 Y` independent and independent of `
If locus i immaterial to fitness can be spliced out
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
The Hyperclimbing Heuristic
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing A Stochastic Search Heuristic
The Setting
0 1` frarr R
f may be stochastic
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing A Stochastic Search Heuristic
The Setting
0 1` frarr R
f may be stochastic
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
The Effect of a Schema Partition
Effect of JIK Variance of average fitness of schemata in JIK
If I sub I prime then Effect(I) le Effect(I prime)
JIK
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
The Effect of a Schema Partition
Effect of JIK Variance of average fitness of schemata in JIK
If I sub I prime then Effect(I) le Effect(I prime)
JIK JI primeK
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption is Weak
Staggered coarse conditional effects assumption is weakWeaker than fitness distribution assumption in the BBH
BBH assumes unstaggered coarse unconditional effects
Weaker assumptions are more likely to be satisfied in practice
Good bc we aspire to explain optimization in all GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption is Weak
Staggered coarse conditional effects assumption is weakWeaker than fitness distribution assumption in the BBH
BBH assumes unstaggered coarse unconditional effects
Weaker assumptions are more likely to be satisfied in practice
Good bc we aspire to explain optimization in all GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Proof Of Concept
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
fitness(x) simN (+025 1) if xi1 oplus xi2 oplus xi3 oplus xi4 = 1N (minus025 1) otherwise
0 Fitness
Pro
babi
lity
minus025 +025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
darr darr darr darr10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
uarr uarr uarr uarrEffective Loci
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Naive approach
Visit loci in groups of four
Check for differentiation inthe multivariate marginalfitness values
Time complexity is Ω(`4)
xi1 xi2 xi3 xi4
ExpectedMarginalFitness
0 0 0 0 minus0250 0 0 1 +0250 0 1 0 +0250 0 1 1 minus0250 1 0 0 +0250 1 0 1 minus0250 1 1 0 minus0250 1 1 1 +0251 0 0 0 +0251 0 0 1 minus0251 0 1 0 minus0251 0 1 1 +0251 1 0 0 minus0251 1 0 1 +0251 1 1 0 +0251 1 1 1 minus025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Visiting loci in groups ofthree or less wonrsquot work
xj
ExpectedMarginalFitness
0 001 00
xj1 xj2
ExpectedMarginalFitness
0 0 000 1 001 0 001 1 00
xj1 xj2 xj3
ExpectedMarginalFitness
0 0 0 000 0 1 000 1 0 000 1 1 001 0 0 001 0 1 001 1 0 001 1 1 00
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=200
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity200AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Symmetry Analytic Conclusion
Expected generations for the red dots to diverge constant wrt
Location of the red dots
Number of blue dots
Dynamics of a blue dot invariant to
Location of the red dots
Number of blue dots
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=1000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity1000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=10000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity10000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
Staircase function descriptor (m n δ σ ` LV )
L =
3 87 middot middot middot 93
42 58 middot middot middot 72
67 73 middot middot middot 81
︸ ︷︷ ︸
n
m V =
1 0 middot middot middot 11 0 middot middot middot 0
0 1 middot middot middot 1
︸ ︷︷ ︸
n
m
Staircase function is a stochastic function f 0 1` rarr R
0 δ 2δ (mminus1)δ mδ
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
δ = 02 σ = 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredInOrder__ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
trials=100 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
35 25 37 4631 5 32 2021 33 50 4240 27 30 312 14 48 2
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
B
Watch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
0 0 1 01 1 0 01 1 0 11 1 0 10 0 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=randommpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Prediction amp Validation
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
crosstype=2Mut=003Animmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
ctype=2StaircaseClampingAnimmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60000
100
200
300
400
500
600
700
800
900
Generation
Un
mu
tate
d lo
ci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
100
200
300
400
500
600
700
800
900
1000
Generation
Unm
uta
ted loci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Conclusion
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 10
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Popperian Method
Logic behind the Popperian Method Contrapositive
Theory rArr Phenomenon hArr notPhenomenonrArr notTheory
Additional tightening by Popper
Unexpected Phenomenon rarr More credence owed to the theoryeg Gravitational lensing rarr General theory of relativity
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Popperian Method
Logic behind the Popperian Method Contrapositive
Theory rArr Phenomenon hArr notPhenomenonrArr notTheory
Additional tightening by Popper
Unexpected Phenomenon rarr More credence owed to the theory
eg Gravitational lensing rarr General theory of relativity
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Additional Requirements in a Science of EC
Weak assumptions about distribution of fitness
This is just Occamrsquos Razor
Upfront proof of concept
Avoid another rdquoRoyal Roads momentrdquo
(Nice to have) Identification of a core computational efficiency
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Additional Requirements in a Science of EC
Weak assumptions about distribution of fitness
This is just Occamrsquos Razor
Upfront proof of concept
Avoid another rdquoRoyal Roads momentrdquo
(Nice to have) Identification of a core computational efficiency
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Additional Requirements in a Science of EC
Weak assumptions about distribution of fitness
This is just Occamrsquos Razor
Upfront proof of concept
Avoid another rdquoRoyal Roads momentrdquo
(Nice to have) Identification of a core computational efficiency
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Hypothesis
GAs with UX perform optimization by efficientlyimplementing a global search heuristic called hyperclimbing
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Outline
1 Highlight Symmetry of UX
2 Describe Hyperclimbing Heuristic
3 Provide proof-of-concept
Show a GA implementing hyperclimbing efficiently
4 Make a prediction and validate it on
MAX-3SATSherrington Kirkpatrick Spin Glasses problem
5 Outline future work
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0
1 0 0 1 1 0 0 1 1 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0X1 X2 X3 X4 X5 X6 X7 X8 X9 X`1 0 0 1 1 0 0 1 1 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0uarr darr darr uarr uarr darr darr darr uarr uarr1 0 0 1 1 0 0 1 1 1
0 0 0 0 1 0 0 1 1 0
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0uarr darr darr uarr uarr darr darr darr uarr uarr1 0 0 1 1 0 0 1 1 1
0 0 0 0 1 0 0 1 1 0Y1 Y2 Y3 Y4 Y5 Y6 Y7 Y8 Y9 Y`
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0uarr darr darr uarr uarr darr darr darr uarr uarr1 0 0 1 1 0 0 1 1 1
0 0 0 0 1 0 0 1 1 0dArr dArr dArr z dArr dArr dArr z dArr dArr0 0 0 1 1 0 0 0 1 0
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
X1 X2 X3 X4 X5 X6 X7 X8 X9 X`
Y1 Y2 Y3 Y4 Y5 Y6 Y7 Y8 Y9 Y`
UX X1 X` are independent
Absence of positional bias (Eshelman et al 1989)
Attributes can be arbitrarily permuted
Suppose Y1 Y` independent and independent of `
If locus i immaterial to fitness can be spliced out
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
The Hyperclimbing Heuristic
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing A Stochastic Search Heuristic
The Setting
0 1` frarr R
f may be stochastic
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing A Stochastic Search Heuristic
The Setting
0 1` frarr R
f may be stochastic
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
The Effect of a Schema Partition
Effect of JIK Variance of average fitness of schemata in JIK
If I sub I prime then Effect(I) le Effect(I prime)
JIK
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
The Effect of a Schema Partition
Effect of JIK Variance of average fitness of schemata in JIK
If I sub I prime then Effect(I) le Effect(I prime)
JIK JI primeK
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption is Weak
Staggered coarse conditional effects assumption is weakWeaker than fitness distribution assumption in the BBH
BBH assumes unstaggered coarse unconditional effects
Weaker assumptions are more likely to be satisfied in practice
Good bc we aspire to explain optimization in all GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption is Weak
Staggered coarse conditional effects assumption is weakWeaker than fitness distribution assumption in the BBH
BBH assumes unstaggered coarse unconditional effects
Weaker assumptions are more likely to be satisfied in practice
Good bc we aspire to explain optimization in all GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Proof Of Concept
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
fitness(x) simN (+025 1) if xi1 oplus xi2 oplus xi3 oplus xi4 = 1N (minus025 1) otherwise
0 Fitness
Pro
babi
lity
minus025 +025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
darr darr darr darr10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
uarr uarr uarr uarrEffective Loci
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Naive approach
Visit loci in groups of four
Check for differentiation inthe multivariate marginalfitness values
Time complexity is Ω(`4)
xi1 xi2 xi3 xi4
ExpectedMarginalFitness
0 0 0 0 minus0250 0 0 1 +0250 0 1 0 +0250 0 1 1 minus0250 1 0 0 +0250 1 0 1 minus0250 1 1 0 minus0250 1 1 1 +0251 0 0 0 +0251 0 0 1 minus0251 0 1 0 minus0251 0 1 1 +0251 1 0 0 minus0251 1 0 1 +0251 1 1 0 +0251 1 1 1 minus025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Visiting loci in groups ofthree or less wonrsquot work
xj
ExpectedMarginalFitness
0 001 00
xj1 xj2
ExpectedMarginalFitness
0 0 000 1 001 0 001 1 00
xj1 xj2 xj3
ExpectedMarginalFitness
0 0 0 000 0 1 000 1 0 000 1 1 001 0 0 001 0 1 001 1 0 001 1 1 00
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=200
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity200AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Symmetry Analytic Conclusion
Expected generations for the red dots to diverge constant wrt
Location of the red dots
Number of blue dots
Dynamics of a blue dot invariant to
Location of the red dots
Number of blue dots
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=1000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity1000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=10000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity10000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
Staircase function descriptor (m n δ σ ` LV )
L =
3 87 middot middot middot 93
42 58 middot middot middot 72
67 73 middot middot middot 81
︸ ︷︷ ︸
n
m V =
1 0 middot middot middot 11 0 middot middot middot 0
0 1 middot middot middot 1
︸ ︷︷ ︸
n
m
Staircase function is a stochastic function f 0 1` rarr R
0 δ 2δ (mminus1)δ mδ
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
δ = 02 σ = 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredInOrder__ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
trials=100 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
35 25 37 4631 5 32 2021 33 50 4240 27 30 312 14 48 2
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
B
Watch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
0 0 1 01 1 0 01 1 0 11 1 0 10 0 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=randommpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Prediction amp Validation
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
crosstype=2Mut=003Animmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
ctype=2StaircaseClampingAnimmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60000
100
200
300
400
500
600
700
800
900
Generation
Un
mu
tate
d lo
ci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
100
200
300
400
500
600
700
800
900
1000
Generation
Unm
uta
ted loci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Conclusion
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 11
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Popperian Method
Logic behind the Popperian Method Contrapositive
Theory rArr Phenomenon hArr notPhenomenonrArr notTheory
Additional tightening by Popper
Unexpected Phenomenon rarr More credence owed to the theory
eg Gravitational lensing rarr General theory of relativity
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Additional Requirements in a Science of EC
Weak assumptions about distribution of fitness
This is just Occamrsquos Razor
Upfront proof of concept
Avoid another rdquoRoyal Roads momentrdquo
(Nice to have) Identification of a core computational efficiency
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Additional Requirements in a Science of EC
Weak assumptions about distribution of fitness
This is just Occamrsquos Razor
Upfront proof of concept
Avoid another rdquoRoyal Roads momentrdquo
(Nice to have) Identification of a core computational efficiency
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Additional Requirements in a Science of EC
Weak assumptions about distribution of fitness
This is just Occamrsquos Razor
Upfront proof of concept
Avoid another rdquoRoyal Roads momentrdquo
(Nice to have) Identification of a core computational efficiency
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Hypothesis
GAs with UX perform optimization by efficientlyimplementing a global search heuristic called hyperclimbing
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Outline
1 Highlight Symmetry of UX
2 Describe Hyperclimbing Heuristic
3 Provide proof-of-concept
Show a GA implementing hyperclimbing efficiently
4 Make a prediction and validate it on
MAX-3SATSherrington Kirkpatrick Spin Glasses problem
5 Outline future work
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0
1 0 0 1 1 0 0 1 1 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0X1 X2 X3 X4 X5 X6 X7 X8 X9 X`1 0 0 1 1 0 0 1 1 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0uarr darr darr uarr uarr darr darr darr uarr uarr1 0 0 1 1 0 0 1 1 1
0 0 0 0 1 0 0 1 1 0
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0uarr darr darr uarr uarr darr darr darr uarr uarr1 0 0 1 1 0 0 1 1 1
0 0 0 0 1 0 0 1 1 0Y1 Y2 Y3 Y4 Y5 Y6 Y7 Y8 Y9 Y`
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0uarr darr darr uarr uarr darr darr darr uarr uarr1 0 0 1 1 0 0 1 1 1
0 0 0 0 1 0 0 1 1 0dArr dArr dArr z dArr dArr dArr z dArr dArr0 0 0 1 1 0 0 0 1 0
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
X1 X2 X3 X4 X5 X6 X7 X8 X9 X`
Y1 Y2 Y3 Y4 Y5 Y6 Y7 Y8 Y9 Y`
UX X1 X` are independent
Absence of positional bias (Eshelman et al 1989)
Attributes can be arbitrarily permuted
Suppose Y1 Y` independent and independent of `
If locus i immaterial to fitness can be spliced out
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
The Hyperclimbing Heuristic
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing A Stochastic Search Heuristic
The Setting
0 1` frarr R
f may be stochastic
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing A Stochastic Search Heuristic
The Setting
0 1` frarr R
f may be stochastic
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
The Effect of a Schema Partition
Effect of JIK Variance of average fitness of schemata in JIK
If I sub I prime then Effect(I) le Effect(I prime)
JIK
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
The Effect of a Schema Partition
Effect of JIK Variance of average fitness of schemata in JIK
If I sub I prime then Effect(I) le Effect(I prime)
JIK JI primeK
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption is Weak
Staggered coarse conditional effects assumption is weakWeaker than fitness distribution assumption in the BBH
BBH assumes unstaggered coarse unconditional effects
Weaker assumptions are more likely to be satisfied in practice
Good bc we aspire to explain optimization in all GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption is Weak
Staggered coarse conditional effects assumption is weakWeaker than fitness distribution assumption in the BBH
BBH assumes unstaggered coarse unconditional effects
Weaker assumptions are more likely to be satisfied in practice
Good bc we aspire to explain optimization in all GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Proof Of Concept
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
fitness(x) simN (+025 1) if xi1 oplus xi2 oplus xi3 oplus xi4 = 1N (minus025 1) otherwise
0 Fitness
Pro
babi
lity
minus025 +025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
darr darr darr darr10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
uarr uarr uarr uarrEffective Loci
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Naive approach
Visit loci in groups of four
Check for differentiation inthe multivariate marginalfitness values
Time complexity is Ω(`4)
xi1 xi2 xi3 xi4
ExpectedMarginalFitness
0 0 0 0 minus0250 0 0 1 +0250 0 1 0 +0250 0 1 1 minus0250 1 0 0 +0250 1 0 1 minus0250 1 1 0 minus0250 1 1 1 +0251 0 0 0 +0251 0 0 1 minus0251 0 1 0 minus0251 0 1 1 +0251 1 0 0 minus0251 1 0 1 +0251 1 1 0 +0251 1 1 1 minus025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Visiting loci in groups ofthree or less wonrsquot work
xj
ExpectedMarginalFitness
0 001 00
xj1 xj2
ExpectedMarginalFitness
0 0 000 1 001 0 001 1 00
xj1 xj2 xj3
ExpectedMarginalFitness
0 0 0 000 0 1 000 1 0 000 1 1 001 0 0 001 0 1 001 1 0 001 1 1 00
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=200
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity200AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Symmetry Analytic Conclusion
Expected generations for the red dots to diverge constant wrt
Location of the red dots
Number of blue dots
Dynamics of a blue dot invariant to
Location of the red dots
Number of blue dots
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=1000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity1000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=10000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity10000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
Staircase function descriptor (m n δ σ ` LV )
L =
3 87 middot middot middot 93
42 58 middot middot middot 72
67 73 middot middot middot 81
︸ ︷︷ ︸
n
m V =
1 0 middot middot middot 11 0 middot middot middot 0
0 1 middot middot middot 1
︸ ︷︷ ︸
n
m
Staircase function is a stochastic function f 0 1` rarr R
0 δ 2δ (mminus1)δ mδ
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
δ = 02 σ = 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredInOrder__ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
trials=100 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
35 25 37 4631 5 32 2021 33 50 4240 27 30 312 14 48 2
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
B
Watch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
0 0 1 01 1 0 01 1 0 11 1 0 10 0 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=randommpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Prediction amp Validation
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
crosstype=2Mut=003Animmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
ctype=2StaircaseClampingAnimmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60000
100
200
300
400
500
600
700
800
900
Generation
Un
mu
tate
d lo
ci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
100
200
300
400
500
600
700
800
900
1000
Generation
Unm
uta
ted loci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Conclusion
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 12
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Additional Requirements in a Science of EC
Weak assumptions about distribution of fitness
This is just Occamrsquos Razor
Upfront proof of concept
Avoid another rdquoRoyal Roads momentrdquo
(Nice to have) Identification of a core computational efficiency
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Additional Requirements in a Science of EC
Weak assumptions about distribution of fitness
This is just Occamrsquos Razor
Upfront proof of concept
Avoid another rdquoRoyal Roads momentrdquo
(Nice to have) Identification of a core computational efficiency
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Additional Requirements in a Science of EC
Weak assumptions about distribution of fitness
This is just Occamrsquos Razor
Upfront proof of concept
Avoid another rdquoRoyal Roads momentrdquo
(Nice to have) Identification of a core computational efficiency
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Hypothesis
GAs with UX perform optimization by efficientlyimplementing a global search heuristic called hyperclimbing
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Outline
1 Highlight Symmetry of UX
2 Describe Hyperclimbing Heuristic
3 Provide proof-of-concept
Show a GA implementing hyperclimbing efficiently
4 Make a prediction and validate it on
MAX-3SATSherrington Kirkpatrick Spin Glasses problem
5 Outline future work
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0
1 0 0 1 1 0 0 1 1 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0X1 X2 X3 X4 X5 X6 X7 X8 X9 X`1 0 0 1 1 0 0 1 1 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0uarr darr darr uarr uarr darr darr darr uarr uarr1 0 0 1 1 0 0 1 1 1
0 0 0 0 1 0 0 1 1 0
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0uarr darr darr uarr uarr darr darr darr uarr uarr1 0 0 1 1 0 0 1 1 1
0 0 0 0 1 0 0 1 1 0Y1 Y2 Y3 Y4 Y5 Y6 Y7 Y8 Y9 Y`
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0uarr darr darr uarr uarr darr darr darr uarr uarr1 0 0 1 1 0 0 1 1 1
0 0 0 0 1 0 0 1 1 0dArr dArr dArr z dArr dArr dArr z dArr dArr0 0 0 1 1 0 0 0 1 0
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
X1 X2 X3 X4 X5 X6 X7 X8 X9 X`
Y1 Y2 Y3 Y4 Y5 Y6 Y7 Y8 Y9 Y`
UX X1 X` are independent
Absence of positional bias (Eshelman et al 1989)
Attributes can be arbitrarily permuted
Suppose Y1 Y` independent and independent of `
If locus i immaterial to fitness can be spliced out
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
The Hyperclimbing Heuristic
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing A Stochastic Search Heuristic
The Setting
0 1` frarr R
f may be stochastic
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing A Stochastic Search Heuristic
The Setting
0 1` frarr R
f may be stochastic
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
The Effect of a Schema Partition
Effect of JIK Variance of average fitness of schemata in JIK
If I sub I prime then Effect(I) le Effect(I prime)
JIK
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
The Effect of a Schema Partition
Effect of JIK Variance of average fitness of schemata in JIK
If I sub I prime then Effect(I) le Effect(I prime)
JIK JI primeK
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption is Weak
Staggered coarse conditional effects assumption is weakWeaker than fitness distribution assumption in the BBH
BBH assumes unstaggered coarse unconditional effects
Weaker assumptions are more likely to be satisfied in practice
Good bc we aspire to explain optimization in all GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption is Weak
Staggered coarse conditional effects assumption is weakWeaker than fitness distribution assumption in the BBH
BBH assumes unstaggered coarse unconditional effects
Weaker assumptions are more likely to be satisfied in practice
Good bc we aspire to explain optimization in all GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Proof Of Concept
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
fitness(x) simN (+025 1) if xi1 oplus xi2 oplus xi3 oplus xi4 = 1N (minus025 1) otherwise
0 Fitness
Pro
babi
lity
minus025 +025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
darr darr darr darr10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
uarr uarr uarr uarrEffective Loci
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Naive approach
Visit loci in groups of four
Check for differentiation inthe multivariate marginalfitness values
Time complexity is Ω(`4)
xi1 xi2 xi3 xi4
ExpectedMarginalFitness
0 0 0 0 minus0250 0 0 1 +0250 0 1 0 +0250 0 1 1 minus0250 1 0 0 +0250 1 0 1 minus0250 1 1 0 minus0250 1 1 1 +0251 0 0 0 +0251 0 0 1 minus0251 0 1 0 minus0251 0 1 1 +0251 1 0 0 minus0251 1 0 1 +0251 1 1 0 +0251 1 1 1 minus025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Visiting loci in groups ofthree or less wonrsquot work
xj
ExpectedMarginalFitness
0 001 00
xj1 xj2
ExpectedMarginalFitness
0 0 000 1 001 0 001 1 00
xj1 xj2 xj3
ExpectedMarginalFitness
0 0 0 000 0 1 000 1 0 000 1 1 001 0 0 001 0 1 001 1 0 001 1 1 00
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=200
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity200AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Symmetry Analytic Conclusion
Expected generations for the red dots to diverge constant wrt
Location of the red dots
Number of blue dots
Dynamics of a blue dot invariant to
Location of the red dots
Number of blue dots
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=1000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity1000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=10000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity10000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
Staircase function descriptor (m n δ σ ` LV )
L =
3 87 middot middot middot 93
42 58 middot middot middot 72
67 73 middot middot middot 81
︸ ︷︷ ︸
n
m V =
1 0 middot middot middot 11 0 middot middot middot 0
0 1 middot middot middot 1
︸ ︷︷ ︸
n
m
Staircase function is a stochastic function f 0 1` rarr R
0 δ 2δ (mminus1)δ mδ
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
δ = 02 σ = 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredInOrder__ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
trials=100 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
35 25 37 4631 5 32 2021 33 50 4240 27 30 312 14 48 2
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
B
Watch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
0 0 1 01 1 0 01 1 0 11 1 0 10 0 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=randommpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Prediction amp Validation
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
crosstype=2Mut=003Animmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
ctype=2StaircaseClampingAnimmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60000
100
200
300
400
500
600
700
800
900
Generation
Un
mu
tate
d lo
ci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
100
200
300
400
500
600
700
800
900
1000
Generation
Unm
uta
ted loci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Conclusion
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 13
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Additional Requirements in a Science of EC
Weak assumptions about distribution of fitness
This is just Occamrsquos Razor
Upfront proof of concept
Avoid another rdquoRoyal Roads momentrdquo
(Nice to have) Identification of a core computational efficiency
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Additional Requirements in a Science of EC
Weak assumptions about distribution of fitness
This is just Occamrsquos Razor
Upfront proof of concept
Avoid another rdquoRoyal Roads momentrdquo
(Nice to have) Identification of a core computational efficiency
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Hypothesis
GAs with UX perform optimization by efficientlyimplementing a global search heuristic called hyperclimbing
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Outline
1 Highlight Symmetry of UX
2 Describe Hyperclimbing Heuristic
3 Provide proof-of-concept
Show a GA implementing hyperclimbing efficiently
4 Make a prediction and validate it on
MAX-3SATSherrington Kirkpatrick Spin Glasses problem
5 Outline future work
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0
1 0 0 1 1 0 0 1 1 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0X1 X2 X3 X4 X5 X6 X7 X8 X9 X`1 0 0 1 1 0 0 1 1 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0uarr darr darr uarr uarr darr darr darr uarr uarr1 0 0 1 1 0 0 1 1 1
0 0 0 0 1 0 0 1 1 0
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0uarr darr darr uarr uarr darr darr darr uarr uarr1 0 0 1 1 0 0 1 1 1
0 0 0 0 1 0 0 1 1 0Y1 Y2 Y3 Y4 Y5 Y6 Y7 Y8 Y9 Y`
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0uarr darr darr uarr uarr darr darr darr uarr uarr1 0 0 1 1 0 0 1 1 1
0 0 0 0 1 0 0 1 1 0dArr dArr dArr z dArr dArr dArr z dArr dArr0 0 0 1 1 0 0 0 1 0
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
X1 X2 X3 X4 X5 X6 X7 X8 X9 X`
Y1 Y2 Y3 Y4 Y5 Y6 Y7 Y8 Y9 Y`
UX X1 X` are independent
Absence of positional bias (Eshelman et al 1989)
Attributes can be arbitrarily permuted
Suppose Y1 Y` independent and independent of `
If locus i immaterial to fitness can be spliced out
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
The Hyperclimbing Heuristic
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing A Stochastic Search Heuristic
The Setting
0 1` frarr R
f may be stochastic
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing A Stochastic Search Heuristic
The Setting
0 1` frarr R
f may be stochastic
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
The Effect of a Schema Partition
Effect of JIK Variance of average fitness of schemata in JIK
If I sub I prime then Effect(I) le Effect(I prime)
JIK
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
The Effect of a Schema Partition
Effect of JIK Variance of average fitness of schemata in JIK
If I sub I prime then Effect(I) le Effect(I prime)
JIK JI primeK
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption is Weak
Staggered coarse conditional effects assumption is weakWeaker than fitness distribution assumption in the BBH
BBH assumes unstaggered coarse unconditional effects
Weaker assumptions are more likely to be satisfied in practice
Good bc we aspire to explain optimization in all GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption is Weak
Staggered coarse conditional effects assumption is weakWeaker than fitness distribution assumption in the BBH
BBH assumes unstaggered coarse unconditional effects
Weaker assumptions are more likely to be satisfied in practice
Good bc we aspire to explain optimization in all GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Proof Of Concept
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
fitness(x) simN (+025 1) if xi1 oplus xi2 oplus xi3 oplus xi4 = 1N (minus025 1) otherwise
0 Fitness
Pro
babi
lity
minus025 +025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
darr darr darr darr10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
uarr uarr uarr uarrEffective Loci
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Naive approach
Visit loci in groups of four
Check for differentiation inthe multivariate marginalfitness values
Time complexity is Ω(`4)
xi1 xi2 xi3 xi4
ExpectedMarginalFitness
0 0 0 0 minus0250 0 0 1 +0250 0 1 0 +0250 0 1 1 minus0250 1 0 0 +0250 1 0 1 minus0250 1 1 0 minus0250 1 1 1 +0251 0 0 0 +0251 0 0 1 minus0251 0 1 0 minus0251 0 1 1 +0251 1 0 0 minus0251 1 0 1 +0251 1 1 0 +0251 1 1 1 minus025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Visiting loci in groups ofthree or less wonrsquot work
xj
ExpectedMarginalFitness
0 001 00
xj1 xj2
ExpectedMarginalFitness
0 0 000 1 001 0 001 1 00
xj1 xj2 xj3
ExpectedMarginalFitness
0 0 0 000 0 1 000 1 0 000 1 1 001 0 0 001 0 1 001 1 0 001 1 1 00
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=200
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity200AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Symmetry Analytic Conclusion
Expected generations for the red dots to diverge constant wrt
Location of the red dots
Number of blue dots
Dynamics of a blue dot invariant to
Location of the red dots
Number of blue dots
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=1000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity1000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=10000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity10000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
Staircase function descriptor (m n δ σ ` LV )
L =
3 87 middot middot middot 93
42 58 middot middot middot 72
67 73 middot middot middot 81
︸ ︷︷ ︸
n
m V =
1 0 middot middot middot 11 0 middot middot middot 0
0 1 middot middot middot 1
︸ ︷︷ ︸
n
m
Staircase function is a stochastic function f 0 1` rarr R
0 δ 2δ (mminus1)δ mδ
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
δ = 02 σ = 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredInOrder__ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
trials=100 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
35 25 37 4631 5 32 2021 33 50 4240 27 30 312 14 48 2
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
B
Watch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
0 0 1 01 1 0 01 1 0 11 1 0 10 0 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=randommpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Prediction amp Validation
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
crosstype=2Mut=003Animmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
ctype=2StaircaseClampingAnimmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60000
100
200
300
400
500
600
700
800
900
Generation
Un
mu
tate
d lo
ci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
100
200
300
400
500
600
700
800
900
1000
Generation
Unm
uta
ted loci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Conclusion
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 14
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Additional Requirements in a Science of EC
Weak assumptions about distribution of fitness
This is just Occamrsquos Razor
Upfront proof of concept
Avoid another rdquoRoyal Roads momentrdquo
(Nice to have) Identification of a core computational efficiency
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Hypothesis
GAs with UX perform optimization by efficientlyimplementing a global search heuristic called hyperclimbing
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Outline
1 Highlight Symmetry of UX
2 Describe Hyperclimbing Heuristic
3 Provide proof-of-concept
Show a GA implementing hyperclimbing efficiently
4 Make a prediction and validate it on
MAX-3SATSherrington Kirkpatrick Spin Glasses problem
5 Outline future work
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0
1 0 0 1 1 0 0 1 1 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0X1 X2 X3 X4 X5 X6 X7 X8 X9 X`1 0 0 1 1 0 0 1 1 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0uarr darr darr uarr uarr darr darr darr uarr uarr1 0 0 1 1 0 0 1 1 1
0 0 0 0 1 0 0 1 1 0
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0uarr darr darr uarr uarr darr darr darr uarr uarr1 0 0 1 1 0 0 1 1 1
0 0 0 0 1 0 0 1 1 0Y1 Y2 Y3 Y4 Y5 Y6 Y7 Y8 Y9 Y`
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0uarr darr darr uarr uarr darr darr darr uarr uarr1 0 0 1 1 0 0 1 1 1
0 0 0 0 1 0 0 1 1 0dArr dArr dArr z dArr dArr dArr z dArr dArr0 0 0 1 1 0 0 0 1 0
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
X1 X2 X3 X4 X5 X6 X7 X8 X9 X`
Y1 Y2 Y3 Y4 Y5 Y6 Y7 Y8 Y9 Y`
UX X1 X` are independent
Absence of positional bias (Eshelman et al 1989)
Attributes can be arbitrarily permuted
Suppose Y1 Y` independent and independent of `
If locus i immaterial to fitness can be spliced out
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
The Hyperclimbing Heuristic
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing A Stochastic Search Heuristic
The Setting
0 1` frarr R
f may be stochastic
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing A Stochastic Search Heuristic
The Setting
0 1` frarr R
f may be stochastic
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
The Effect of a Schema Partition
Effect of JIK Variance of average fitness of schemata in JIK
If I sub I prime then Effect(I) le Effect(I prime)
JIK
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
The Effect of a Schema Partition
Effect of JIK Variance of average fitness of schemata in JIK
If I sub I prime then Effect(I) le Effect(I prime)
JIK JI primeK
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption is Weak
Staggered coarse conditional effects assumption is weakWeaker than fitness distribution assumption in the BBH
BBH assumes unstaggered coarse unconditional effects
Weaker assumptions are more likely to be satisfied in practice
Good bc we aspire to explain optimization in all GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption is Weak
Staggered coarse conditional effects assumption is weakWeaker than fitness distribution assumption in the BBH
BBH assumes unstaggered coarse unconditional effects
Weaker assumptions are more likely to be satisfied in practice
Good bc we aspire to explain optimization in all GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Proof Of Concept
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
fitness(x) simN (+025 1) if xi1 oplus xi2 oplus xi3 oplus xi4 = 1N (minus025 1) otherwise
0 Fitness
Pro
babi
lity
minus025 +025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
darr darr darr darr10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
uarr uarr uarr uarrEffective Loci
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Naive approach
Visit loci in groups of four
Check for differentiation inthe multivariate marginalfitness values
Time complexity is Ω(`4)
xi1 xi2 xi3 xi4
ExpectedMarginalFitness
0 0 0 0 minus0250 0 0 1 +0250 0 1 0 +0250 0 1 1 minus0250 1 0 0 +0250 1 0 1 minus0250 1 1 0 minus0250 1 1 1 +0251 0 0 0 +0251 0 0 1 minus0251 0 1 0 minus0251 0 1 1 +0251 1 0 0 minus0251 1 0 1 +0251 1 1 0 +0251 1 1 1 minus025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Visiting loci in groups ofthree or less wonrsquot work
xj
ExpectedMarginalFitness
0 001 00
xj1 xj2
ExpectedMarginalFitness
0 0 000 1 001 0 001 1 00
xj1 xj2 xj3
ExpectedMarginalFitness
0 0 0 000 0 1 000 1 0 000 1 1 001 0 0 001 0 1 001 1 0 001 1 1 00
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=200
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity200AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Symmetry Analytic Conclusion
Expected generations for the red dots to diverge constant wrt
Location of the red dots
Number of blue dots
Dynamics of a blue dot invariant to
Location of the red dots
Number of blue dots
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=1000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity1000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=10000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity10000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
Staircase function descriptor (m n δ σ ` LV )
L =
3 87 middot middot middot 93
42 58 middot middot middot 72
67 73 middot middot middot 81
︸ ︷︷ ︸
n
m V =
1 0 middot middot middot 11 0 middot middot middot 0
0 1 middot middot middot 1
︸ ︷︷ ︸
n
m
Staircase function is a stochastic function f 0 1` rarr R
0 δ 2δ (mminus1)δ mδ
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
δ = 02 σ = 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredInOrder__ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
trials=100 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
35 25 37 4631 5 32 2021 33 50 4240 27 30 312 14 48 2
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
B
Watch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
0 0 1 01 1 0 01 1 0 11 1 0 10 0 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=randommpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Prediction amp Validation
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
crosstype=2Mut=003Animmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
ctype=2StaircaseClampingAnimmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60000
100
200
300
400
500
600
700
800
900
Generation
Un
mu
tate
d lo
ci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
100
200
300
400
500
600
700
800
900
1000
Generation
Unm
uta
ted loci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Conclusion
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 15
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Hypothesis
GAs with UX perform optimization by efficientlyimplementing a global search heuristic called hyperclimbing
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Outline
1 Highlight Symmetry of UX
2 Describe Hyperclimbing Heuristic
3 Provide proof-of-concept
Show a GA implementing hyperclimbing efficiently
4 Make a prediction and validate it on
MAX-3SATSherrington Kirkpatrick Spin Glasses problem
5 Outline future work
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0
1 0 0 1 1 0 0 1 1 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0X1 X2 X3 X4 X5 X6 X7 X8 X9 X`1 0 0 1 1 0 0 1 1 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0uarr darr darr uarr uarr darr darr darr uarr uarr1 0 0 1 1 0 0 1 1 1
0 0 0 0 1 0 0 1 1 0
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0uarr darr darr uarr uarr darr darr darr uarr uarr1 0 0 1 1 0 0 1 1 1
0 0 0 0 1 0 0 1 1 0Y1 Y2 Y3 Y4 Y5 Y6 Y7 Y8 Y9 Y`
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0uarr darr darr uarr uarr darr darr darr uarr uarr1 0 0 1 1 0 0 1 1 1
0 0 0 0 1 0 0 1 1 0dArr dArr dArr z dArr dArr dArr z dArr dArr0 0 0 1 1 0 0 0 1 0
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
X1 X2 X3 X4 X5 X6 X7 X8 X9 X`
Y1 Y2 Y3 Y4 Y5 Y6 Y7 Y8 Y9 Y`
UX X1 X` are independent
Absence of positional bias (Eshelman et al 1989)
Attributes can be arbitrarily permuted
Suppose Y1 Y` independent and independent of `
If locus i immaterial to fitness can be spliced out
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
The Hyperclimbing Heuristic
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing A Stochastic Search Heuristic
The Setting
0 1` frarr R
f may be stochastic
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing A Stochastic Search Heuristic
The Setting
0 1` frarr R
f may be stochastic
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
The Effect of a Schema Partition
Effect of JIK Variance of average fitness of schemata in JIK
If I sub I prime then Effect(I) le Effect(I prime)
JIK
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
The Effect of a Schema Partition
Effect of JIK Variance of average fitness of schemata in JIK
If I sub I prime then Effect(I) le Effect(I prime)
JIK JI primeK
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption is Weak
Staggered coarse conditional effects assumption is weakWeaker than fitness distribution assumption in the BBH
BBH assumes unstaggered coarse unconditional effects
Weaker assumptions are more likely to be satisfied in practice
Good bc we aspire to explain optimization in all GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption is Weak
Staggered coarse conditional effects assumption is weakWeaker than fitness distribution assumption in the BBH
BBH assumes unstaggered coarse unconditional effects
Weaker assumptions are more likely to be satisfied in practice
Good bc we aspire to explain optimization in all GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Proof Of Concept
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
fitness(x) simN (+025 1) if xi1 oplus xi2 oplus xi3 oplus xi4 = 1N (minus025 1) otherwise
0 Fitness
Pro
babi
lity
minus025 +025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
darr darr darr darr10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
uarr uarr uarr uarrEffective Loci
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Naive approach
Visit loci in groups of four
Check for differentiation inthe multivariate marginalfitness values
Time complexity is Ω(`4)
xi1 xi2 xi3 xi4
ExpectedMarginalFitness
0 0 0 0 minus0250 0 0 1 +0250 0 1 0 +0250 0 1 1 minus0250 1 0 0 +0250 1 0 1 minus0250 1 1 0 minus0250 1 1 1 +0251 0 0 0 +0251 0 0 1 minus0251 0 1 0 minus0251 0 1 1 +0251 1 0 0 minus0251 1 0 1 +0251 1 1 0 +0251 1 1 1 minus025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Visiting loci in groups ofthree or less wonrsquot work
xj
ExpectedMarginalFitness
0 001 00
xj1 xj2
ExpectedMarginalFitness
0 0 000 1 001 0 001 1 00
xj1 xj2 xj3
ExpectedMarginalFitness
0 0 0 000 0 1 000 1 0 000 1 1 001 0 0 001 0 1 001 1 0 001 1 1 00
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=200
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity200AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Symmetry Analytic Conclusion
Expected generations for the red dots to diverge constant wrt
Location of the red dots
Number of blue dots
Dynamics of a blue dot invariant to
Location of the red dots
Number of blue dots
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=1000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity1000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=10000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity10000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
Staircase function descriptor (m n δ σ ` LV )
L =
3 87 middot middot middot 93
42 58 middot middot middot 72
67 73 middot middot middot 81
︸ ︷︷ ︸
n
m V =
1 0 middot middot middot 11 0 middot middot middot 0
0 1 middot middot middot 1
︸ ︷︷ ︸
n
m
Staircase function is a stochastic function f 0 1` rarr R
0 δ 2δ (mminus1)δ mδ
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
δ = 02 σ = 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredInOrder__ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
trials=100 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
35 25 37 4631 5 32 2021 33 50 4240 27 30 312 14 48 2
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
B
Watch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
0 0 1 01 1 0 01 1 0 11 1 0 10 0 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=randommpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Prediction amp Validation
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
crosstype=2Mut=003Animmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
ctype=2StaircaseClampingAnimmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60000
100
200
300
400
500
600
700
800
900
Generation
Un
mu
tate
d lo
ci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
100
200
300
400
500
600
700
800
900
1000
Generation
Unm
uta
ted loci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Conclusion
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 16
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Outline
1 Highlight Symmetry of UX
2 Describe Hyperclimbing Heuristic
3 Provide proof-of-concept
Show a GA implementing hyperclimbing efficiently
4 Make a prediction and validate it on
MAX-3SATSherrington Kirkpatrick Spin Glasses problem
5 Outline future work
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0
1 0 0 1 1 0 0 1 1 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0X1 X2 X3 X4 X5 X6 X7 X8 X9 X`1 0 0 1 1 0 0 1 1 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0uarr darr darr uarr uarr darr darr darr uarr uarr1 0 0 1 1 0 0 1 1 1
0 0 0 0 1 0 0 1 1 0
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0uarr darr darr uarr uarr darr darr darr uarr uarr1 0 0 1 1 0 0 1 1 1
0 0 0 0 1 0 0 1 1 0Y1 Y2 Y3 Y4 Y5 Y6 Y7 Y8 Y9 Y`
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0uarr darr darr uarr uarr darr darr darr uarr uarr1 0 0 1 1 0 0 1 1 1
0 0 0 0 1 0 0 1 1 0dArr dArr dArr z dArr dArr dArr z dArr dArr0 0 0 1 1 0 0 0 1 0
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
X1 X2 X3 X4 X5 X6 X7 X8 X9 X`
Y1 Y2 Y3 Y4 Y5 Y6 Y7 Y8 Y9 Y`
UX X1 X` are independent
Absence of positional bias (Eshelman et al 1989)
Attributes can be arbitrarily permuted
Suppose Y1 Y` independent and independent of `
If locus i immaterial to fitness can be spliced out
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
The Hyperclimbing Heuristic
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing A Stochastic Search Heuristic
The Setting
0 1` frarr R
f may be stochastic
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing A Stochastic Search Heuristic
The Setting
0 1` frarr R
f may be stochastic
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
The Effect of a Schema Partition
Effect of JIK Variance of average fitness of schemata in JIK
If I sub I prime then Effect(I) le Effect(I prime)
JIK
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
The Effect of a Schema Partition
Effect of JIK Variance of average fitness of schemata in JIK
If I sub I prime then Effect(I) le Effect(I prime)
JIK JI primeK
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption is Weak
Staggered coarse conditional effects assumption is weakWeaker than fitness distribution assumption in the BBH
BBH assumes unstaggered coarse unconditional effects
Weaker assumptions are more likely to be satisfied in practice
Good bc we aspire to explain optimization in all GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption is Weak
Staggered coarse conditional effects assumption is weakWeaker than fitness distribution assumption in the BBH
BBH assumes unstaggered coarse unconditional effects
Weaker assumptions are more likely to be satisfied in practice
Good bc we aspire to explain optimization in all GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Proof Of Concept
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
fitness(x) simN (+025 1) if xi1 oplus xi2 oplus xi3 oplus xi4 = 1N (minus025 1) otherwise
0 Fitness
Pro
babi
lity
minus025 +025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
darr darr darr darr10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
uarr uarr uarr uarrEffective Loci
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Naive approach
Visit loci in groups of four
Check for differentiation inthe multivariate marginalfitness values
Time complexity is Ω(`4)
xi1 xi2 xi3 xi4
ExpectedMarginalFitness
0 0 0 0 minus0250 0 0 1 +0250 0 1 0 +0250 0 1 1 minus0250 1 0 0 +0250 1 0 1 minus0250 1 1 0 minus0250 1 1 1 +0251 0 0 0 +0251 0 0 1 minus0251 0 1 0 minus0251 0 1 1 +0251 1 0 0 minus0251 1 0 1 +0251 1 1 0 +0251 1 1 1 minus025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Visiting loci in groups ofthree or less wonrsquot work
xj
ExpectedMarginalFitness
0 001 00
xj1 xj2
ExpectedMarginalFitness
0 0 000 1 001 0 001 1 00
xj1 xj2 xj3
ExpectedMarginalFitness
0 0 0 000 0 1 000 1 0 000 1 1 001 0 0 001 0 1 001 1 0 001 1 1 00
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=200
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity200AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Symmetry Analytic Conclusion
Expected generations for the red dots to diverge constant wrt
Location of the red dots
Number of blue dots
Dynamics of a blue dot invariant to
Location of the red dots
Number of blue dots
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=1000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity1000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=10000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity10000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
Staircase function descriptor (m n δ σ ` LV )
L =
3 87 middot middot middot 93
42 58 middot middot middot 72
67 73 middot middot middot 81
︸ ︷︷ ︸
n
m V =
1 0 middot middot middot 11 0 middot middot middot 0
0 1 middot middot middot 1
︸ ︷︷ ︸
n
m
Staircase function is a stochastic function f 0 1` rarr R
0 δ 2δ (mminus1)δ mδ
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
δ = 02 σ = 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredInOrder__ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
trials=100 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
35 25 37 4631 5 32 2021 33 50 4240 27 30 312 14 48 2
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
B
Watch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
0 0 1 01 1 0 01 1 0 11 1 0 10 0 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=randommpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Prediction amp Validation
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
crosstype=2Mut=003Animmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
ctype=2StaircaseClampingAnimmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60000
100
200
300
400
500
600
700
800
900
Generation
Un
mu
tate
d lo
ci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
100
200
300
400
500
600
700
800
900
1000
Generation
Unm
uta
ted loci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Conclusion
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 17
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0
1 0 0 1 1 0 0 1 1 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0X1 X2 X3 X4 X5 X6 X7 X8 X9 X`1 0 0 1 1 0 0 1 1 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0uarr darr darr uarr uarr darr darr darr uarr uarr1 0 0 1 1 0 0 1 1 1
0 0 0 0 1 0 0 1 1 0
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0uarr darr darr uarr uarr darr darr darr uarr uarr1 0 0 1 1 0 0 1 1 1
0 0 0 0 1 0 0 1 1 0Y1 Y2 Y3 Y4 Y5 Y6 Y7 Y8 Y9 Y`
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0uarr darr darr uarr uarr darr darr darr uarr uarr1 0 0 1 1 0 0 1 1 1
0 0 0 0 1 0 0 1 1 0dArr dArr dArr z dArr dArr dArr z dArr dArr0 0 0 1 1 0 0 0 1 0
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
X1 X2 X3 X4 X5 X6 X7 X8 X9 X`
Y1 Y2 Y3 Y4 Y5 Y6 Y7 Y8 Y9 Y`
UX X1 X` are independent
Absence of positional bias (Eshelman et al 1989)
Attributes can be arbitrarily permuted
Suppose Y1 Y` independent and independent of `
If locus i immaterial to fitness can be spliced out
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
The Hyperclimbing Heuristic
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing A Stochastic Search Heuristic
The Setting
0 1` frarr R
f may be stochastic
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing A Stochastic Search Heuristic
The Setting
0 1` frarr R
f may be stochastic
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
The Effect of a Schema Partition
Effect of JIK Variance of average fitness of schemata in JIK
If I sub I prime then Effect(I) le Effect(I prime)
JIK
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
The Effect of a Schema Partition
Effect of JIK Variance of average fitness of schemata in JIK
If I sub I prime then Effect(I) le Effect(I prime)
JIK JI primeK
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption is Weak
Staggered coarse conditional effects assumption is weakWeaker than fitness distribution assumption in the BBH
BBH assumes unstaggered coarse unconditional effects
Weaker assumptions are more likely to be satisfied in practice
Good bc we aspire to explain optimization in all GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption is Weak
Staggered coarse conditional effects assumption is weakWeaker than fitness distribution assumption in the BBH
BBH assumes unstaggered coarse unconditional effects
Weaker assumptions are more likely to be satisfied in practice
Good bc we aspire to explain optimization in all GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Proof Of Concept
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
fitness(x) simN (+025 1) if xi1 oplus xi2 oplus xi3 oplus xi4 = 1N (minus025 1) otherwise
0 Fitness
Pro
babi
lity
minus025 +025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
darr darr darr darr10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
uarr uarr uarr uarrEffective Loci
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Naive approach
Visit loci in groups of four
Check for differentiation inthe multivariate marginalfitness values
Time complexity is Ω(`4)
xi1 xi2 xi3 xi4
ExpectedMarginalFitness
0 0 0 0 minus0250 0 0 1 +0250 0 1 0 +0250 0 1 1 minus0250 1 0 0 +0250 1 0 1 minus0250 1 1 0 minus0250 1 1 1 +0251 0 0 0 +0251 0 0 1 minus0251 0 1 0 minus0251 0 1 1 +0251 1 0 0 minus0251 1 0 1 +0251 1 1 0 +0251 1 1 1 minus025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Visiting loci in groups ofthree or less wonrsquot work
xj
ExpectedMarginalFitness
0 001 00
xj1 xj2
ExpectedMarginalFitness
0 0 000 1 001 0 001 1 00
xj1 xj2 xj3
ExpectedMarginalFitness
0 0 0 000 0 1 000 1 0 000 1 1 001 0 0 001 0 1 001 1 0 001 1 1 00
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=200
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity200AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Symmetry Analytic Conclusion
Expected generations for the red dots to diverge constant wrt
Location of the red dots
Number of blue dots
Dynamics of a blue dot invariant to
Location of the red dots
Number of blue dots
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=1000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity1000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=10000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity10000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
Staircase function descriptor (m n δ σ ` LV )
L =
3 87 middot middot middot 93
42 58 middot middot middot 72
67 73 middot middot middot 81
︸ ︷︷ ︸
n
m V =
1 0 middot middot middot 11 0 middot middot middot 0
0 1 middot middot middot 1
︸ ︷︷ ︸
n
m
Staircase function is a stochastic function f 0 1` rarr R
0 δ 2δ (mminus1)δ mδ
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
δ = 02 σ = 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredInOrder__ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
trials=100 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
35 25 37 4631 5 32 2021 33 50 4240 27 30 312 14 48 2
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
B
Watch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
0 0 1 01 1 0 01 1 0 11 1 0 10 0 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=randommpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Prediction amp Validation
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
crosstype=2Mut=003Animmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
ctype=2StaircaseClampingAnimmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60000
100
200
300
400
500
600
700
800
900
Generation
Un
mu
tate
d lo
ci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
100
200
300
400
500
600
700
800
900
1000
Generation
Unm
uta
ted loci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Conclusion
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 18
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0X1 X2 X3 X4 X5 X6 X7 X8 X9 X`1 0 0 1 1 0 0 1 1 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0uarr darr darr uarr uarr darr darr darr uarr uarr1 0 0 1 1 0 0 1 1 1
0 0 0 0 1 0 0 1 1 0
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0uarr darr darr uarr uarr darr darr darr uarr uarr1 0 0 1 1 0 0 1 1 1
0 0 0 0 1 0 0 1 1 0Y1 Y2 Y3 Y4 Y5 Y6 Y7 Y8 Y9 Y`
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0uarr darr darr uarr uarr darr darr darr uarr uarr1 0 0 1 1 0 0 1 1 1
0 0 0 0 1 0 0 1 1 0dArr dArr dArr z dArr dArr dArr z dArr dArr0 0 0 1 1 0 0 0 1 0
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
X1 X2 X3 X4 X5 X6 X7 X8 X9 X`
Y1 Y2 Y3 Y4 Y5 Y6 Y7 Y8 Y9 Y`
UX X1 X` are independent
Absence of positional bias (Eshelman et al 1989)
Attributes can be arbitrarily permuted
Suppose Y1 Y` independent and independent of `
If locus i immaterial to fitness can be spliced out
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
The Hyperclimbing Heuristic
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing A Stochastic Search Heuristic
The Setting
0 1` frarr R
f may be stochastic
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing A Stochastic Search Heuristic
The Setting
0 1` frarr R
f may be stochastic
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
The Effect of a Schema Partition
Effect of JIK Variance of average fitness of schemata in JIK
If I sub I prime then Effect(I) le Effect(I prime)
JIK
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
The Effect of a Schema Partition
Effect of JIK Variance of average fitness of schemata in JIK
If I sub I prime then Effect(I) le Effect(I prime)
JIK JI primeK
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption is Weak
Staggered coarse conditional effects assumption is weakWeaker than fitness distribution assumption in the BBH
BBH assumes unstaggered coarse unconditional effects
Weaker assumptions are more likely to be satisfied in practice
Good bc we aspire to explain optimization in all GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption is Weak
Staggered coarse conditional effects assumption is weakWeaker than fitness distribution assumption in the BBH
BBH assumes unstaggered coarse unconditional effects
Weaker assumptions are more likely to be satisfied in practice
Good bc we aspire to explain optimization in all GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Proof Of Concept
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
fitness(x) simN (+025 1) if xi1 oplus xi2 oplus xi3 oplus xi4 = 1N (minus025 1) otherwise
0 Fitness
Pro
babi
lity
minus025 +025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
darr darr darr darr10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
uarr uarr uarr uarrEffective Loci
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Naive approach
Visit loci in groups of four
Check for differentiation inthe multivariate marginalfitness values
Time complexity is Ω(`4)
xi1 xi2 xi3 xi4
ExpectedMarginalFitness
0 0 0 0 minus0250 0 0 1 +0250 0 1 0 +0250 0 1 1 minus0250 1 0 0 +0250 1 0 1 minus0250 1 1 0 minus0250 1 1 1 +0251 0 0 0 +0251 0 0 1 minus0251 0 1 0 minus0251 0 1 1 +0251 1 0 0 minus0251 1 0 1 +0251 1 1 0 +0251 1 1 1 minus025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Visiting loci in groups ofthree or less wonrsquot work
xj
ExpectedMarginalFitness
0 001 00
xj1 xj2
ExpectedMarginalFitness
0 0 000 1 001 0 001 1 00
xj1 xj2 xj3
ExpectedMarginalFitness
0 0 0 000 0 1 000 1 0 000 1 1 001 0 0 001 0 1 001 1 0 001 1 1 00
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=200
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity200AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Symmetry Analytic Conclusion
Expected generations for the red dots to diverge constant wrt
Location of the red dots
Number of blue dots
Dynamics of a blue dot invariant to
Location of the red dots
Number of blue dots
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=1000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity1000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=10000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity10000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
Staircase function descriptor (m n δ σ ` LV )
L =
3 87 middot middot middot 93
42 58 middot middot middot 72
67 73 middot middot middot 81
︸ ︷︷ ︸
n
m V =
1 0 middot middot middot 11 0 middot middot middot 0
0 1 middot middot middot 1
︸ ︷︷ ︸
n
m
Staircase function is a stochastic function f 0 1` rarr R
0 δ 2δ (mminus1)δ mδ
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
δ = 02 σ = 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredInOrder__ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
trials=100 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
35 25 37 4631 5 32 2021 33 50 4240 27 30 312 14 48 2
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
B
Watch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
0 0 1 01 1 0 01 1 0 11 1 0 10 0 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=randommpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Prediction amp Validation
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
crosstype=2Mut=003Animmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
ctype=2StaircaseClampingAnimmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60000
100
200
300
400
500
600
700
800
900
Generation
Un
mu
tate
d lo
ci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
100
200
300
400
500
600
700
800
900
1000
Generation
Unm
uta
ted loci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Conclusion
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 19
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0uarr darr darr uarr uarr darr darr darr uarr uarr1 0 0 1 1 0 0 1 1 1
0 0 0 0 1 0 0 1 1 0
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0uarr darr darr uarr uarr darr darr darr uarr uarr1 0 0 1 1 0 0 1 1 1
0 0 0 0 1 0 0 1 1 0Y1 Y2 Y3 Y4 Y5 Y6 Y7 Y8 Y9 Y`
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0uarr darr darr uarr uarr darr darr darr uarr uarr1 0 0 1 1 0 0 1 1 1
0 0 0 0 1 0 0 1 1 0dArr dArr dArr z dArr dArr dArr z dArr dArr0 0 0 1 1 0 0 0 1 0
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
X1 X2 X3 X4 X5 X6 X7 X8 X9 X`
Y1 Y2 Y3 Y4 Y5 Y6 Y7 Y8 Y9 Y`
UX X1 X` are independent
Absence of positional bias (Eshelman et al 1989)
Attributes can be arbitrarily permuted
Suppose Y1 Y` independent and independent of `
If locus i immaterial to fitness can be spliced out
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
The Hyperclimbing Heuristic
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing A Stochastic Search Heuristic
The Setting
0 1` frarr R
f may be stochastic
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing A Stochastic Search Heuristic
The Setting
0 1` frarr R
f may be stochastic
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
The Effect of a Schema Partition
Effect of JIK Variance of average fitness of schemata in JIK
If I sub I prime then Effect(I) le Effect(I prime)
JIK
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
The Effect of a Schema Partition
Effect of JIK Variance of average fitness of schemata in JIK
If I sub I prime then Effect(I) le Effect(I prime)
JIK JI primeK
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption is Weak
Staggered coarse conditional effects assumption is weakWeaker than fitness distribution assumption in the BBH
BBH assumes unstaggered coarse unconditional effects
Weaker assumptions are more likely to be satisfied in practice
Good bc we aspire to explain optimization in all GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption is Weak
Staggered coarse conditional effects assumption is weakWeaker than fitness distribution assumption in the BBH
BBH assumes unstaggered coarse unconditional effects
Weaker assumptions are more likely to be satisfied in practice
Good bc we aspire to explain optimization in all GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Proof Of Concept
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
fitness(x) simN (+025 1) if xi1 oplus xi2 oplus xi3 oplus xi4 = 1N (minus025 1) otherwise
0 Fitness
Pro
babi
lity
minus025 +025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
darr darr darr darr10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
uarr uarr uarr uarrEffective Loci
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Naive approach
Visit loci in groups of four
Check for differentiation inthe multivariate marginalfitness values
Time complexity is Ω(`4)
xi1 xi2 xi3 xi4
ExpectedMarginalFitness
0 0 0 0 minus0250 0 0 1 +0250 0 1 0 +0250 0 1 1 minus0250 1 0 0 +0250 1 0 1 minus0250 1 1 0 minus0250 1 1 1 +0251 0 0 0 +0251 0 0 1 minus0251 0 1 0 minus0251 0 1 1 +0251 1 0 0 minus0251 1 0 1 +0251 1 1 0 +0251 1 1 1 minus025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Visiting loci in groups ofthree or less wonrsquot work
xj
ExpectedMarginalFitness
0 001 00
xj1 xj2
ExpectedMarginalFitness
0 0 000 1 001 0 001 1 00
xj1 xj2 xj3
ExpectedMarginalFitness
0 0 0 000 0 1 000 1 0 000 1 1 001 0 0 001 0 1 001 1 0 001 1 1 00
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=200
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity200AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Symmetry Analytic Conclusion
Expected generations for the red dots to diverge constant wrt
Location of the red dots
Number of blue dots
Dynamics of a blue dot invariant to
Location of the red dots
Number of blue dots
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=1000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity1000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=10000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity10000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
Staircase function descriptor (m n δ σ ` LV )
L =
3 87 middot middot middot 93
42 58 middot middot middot 72
67 73 middot middot middot 81
︸ ︷︷ ︸
n
m V =
1 0 middot middot middot 11 0 middot middot middot 0
0 1 middot middot middot 1
︸ ︷︷ ︸
n
m
Staircase function is a stochastic function f 0 1` rarr R
0 δ 2δ (mminus1)δ mδ
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
δ = 02 σ = 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredInOrder__ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
trials=100 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
35 25 37 4631 5 32 2021 33 50 4240 27 30 312 14 48 2
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
B
Watch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
0 0 1 01 1 0 01 1 0 11 1 0 10 0 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=randommpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Prediction amp Validation
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
crosstype=2Mut=003Animmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
ctype=2StaircaseClampingAnimmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60000
100
200
300
400
500
600
700
800
900
Generation
Un
mu
tate
d lo
ci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
100
200
300
400
500
600
700
800
900
1000
Generation
Unm
uta
ted loci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Conclusion
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 20
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0uarr darr darr uarr uarr darr darr darr uarr uarr1 0 0 1 1 0 0 1 1 1
0 0 0 0 1 0 0 1 1 0Y1 Y2 Y3 Y4 Y5 Y6 Y7 Y8 Y9 Y`
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0uarr darr darr uarr uarr darr darr darr uarr uarr1 0 0 1 1 0 0 1 1 1
0 0 0 0 1 0 0 1 1 0dArr dArr dArr z dArr dArr dArr z dArr dArr0 0 0 1 1 0 0 0 1 0
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
X1 X2 X3 X4 X5 X6 X7 X8 X9 X`
Y1 Y2 Y3 Y4 Y5 Y6 Y7 Y8 Y9 Y`
UX X1 X` are independent
Absence of positional bias (Eshelman et al 1989)
Attributes can be arbitrarily permuted
Suppose Y1 Y` independent and independent of `
If locus i immaterial to fitness can be spliced out
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
The Hyperclimbing Heuristic
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing A Stochastic Search Heuristic
The Setting
0 1` frarr R
f may be stochastic
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing A Stochastic Search Heuristic
The Setting
0 1` frarr R
f may be stochastic
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
The Effect of a Schema Partition
Effect of JIK Variance of average fitness of schemata in JIK
If I sub I prime then Effect(I) le Effect(I prime)
JIK
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
The Effect of a Schema Partition
Effect of JIK Variance of average fitness of schemata in JIK
If I sub I prime then Effect(I) le Effect(I prime)
JIK JI primeK
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption is Weak
Staggered coarse conditional effects assumption is weakWeaker than fitness distribution assumption in the BBH
BBH assumes unstaggered coarse unconditional effects
Weaker assumptions are more likely to be satisfied in practice
Good bc we aspire to explain optimization in all GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption is Weak
Staggered coarse conditional effects assumption is weakWeaker than fitness distribution assumption in the BBH
BBH assumes unstaggered coarse unconditional effects
Weaker assumptions are more likely to be satisfied in practice
Good bc we aspire to explain optimization in all GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Proof Of Concept
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
fitness(x) simN (+025 1) if xi1 oplus xi2 oplus xi3 oplus xi4 = 1N (minus025 1) otherwise
0 Fitness
Pro
babi
lity
minus025 +025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
darr darr darr darr10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
uarr uarr uarr uarrEffective Loci
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Naive approach
Visit loci in groups of four
Check for differentiation inthe multivariate marginalfitness values
Time complexity is Ω(`4)
xi1 xi2 xi3 xi4
ExpectedMarginalFitness
0 0 0 0 minus0250 0 0 1 +0250 0 1 0 +0250 0 1 1 minus0250 1 0 0 +0250 1 0 1 minus0250 1 1 0 minus0250 1 1 1 +0251 0 0 0 +0251 0 0 1 minus0251 0 1 0 minus0251 0 1 1 +0251 1 0 0 minus0251 1 0 1 +0251 1 1 0 +0251 1 1 1 minus025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Visiting loci in groups ofthree or less wonrsquot work
xj
ExpectedMarginalFitness
0 001 00
xj1 xj2
ExpectedMarginalFitness
0 0 000 1 001 0 001 1 00
xj1 xj2 xj3
ExpectedMarginalFitness
0 0 0 000 0 1 000 1 0 000 1 1 001 0 0 001 0 1 001 1 0 001 1 1 00
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=200
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity200AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Symmetry Analytic Conclusion
Expected generations for the red dots to diverge constant wrt
Location of the red dots
Number of blue dots
Dynamics of a blue dot invariant to
Location of the red dots
Number of blue dots
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=1000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity1000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=10000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity10000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
Staircase function descriptor (m n δ σ ` LV )
L =
3 87 middot middot middot 93
42 58 middot middot middot 72
67 73 middot middot middot 81
︸ ︷︷ ︸
n
m V =
1 0 middot middot middot 11 0 middot middot middot 0
0 1 middot middot middot 1
︸ ︷︷ ︸
n
m
Staircase function is a stochastic function f 0 1` rarr R
0 δ 2δ (mminus1)δ mδ
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
δ = 02 σ = 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredInOrder__ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
trials=100 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
35 25 37 4631 5 32 2021 33 50 4240 27 30 312 14 48 2
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
B
Watch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
0 0 1 01 1 0 01 1 0 11 1 0 10 0 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=randommpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Prediction amp Validation
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
crosstype=2Mut=003Animmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
ctype=2StaircaseClampingAnimmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60000
100
200
300
400
500
600
700
800
900
Generation
Un
mu
tate
d lo
ci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
100
200
300
400
500
600
700
800
900
1000
Generation
Unm
uta
ted loci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Conclusion
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 21
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
0 1 0 0 1 0 1 0 1 0uarr darr darr uarr uarr darr darr darr uarr uarr1 0 0 1 1 0 0 1 1 1
0 0 0 0 1 0 0 1 1 0dArr dArr dArr z dArr dArr dArr z dArr dArr0 0 0 1 1 0 0 0 1 0
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
X1 X2 X3 X4 X5 X6 X7 X8 X9 X`
Y1 Y2 Y3 Y4 Y5 Y6 Y7 Y8 Y9 Y`
UX X1 X` are independent
Absence of positional bias (Eshelman et al 1989)
Attributes can be arbitrarily permuted
Suppose Y1 Y` independent and independent of `
If locus i immaterial to fitness can be spliced out
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
The Hyperclimbing Heuristic
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing A Stochastic Search Heuristic
The Setting
0 1` frarr R
f may be stochastic
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing A Stochastic Search Heuristic
The Setting
0 1` frarr R
f may be stochastic
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
The Effect of a Schema Partition
Effect of JIK Variance of average fitness of schemata in JIK
If I sub I prime then Effect(I) le Effect(I prime)
JIK
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
The Effect of a Schema Partition
Effect of JIK Variance of average fitness of schemata in JIK
If I sub I prime then Effect(I) le Effect(I prime)
JIK JI primeK
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption is Weak
Staggered coarse conditional effects assumption is weakWeaker than fitness distribution assumption in the BBH
BBH assumes unstaggered coarse unconditional effects
Weaker assumptions are more likely to be satisfied in practice
Good bc we aspire to explain optimization in all GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption is Weak
Staggered coarse conditional effects assumption is weakWeaker than fitness distribution assumption in the BBH
BBH assumes unstaggered coarse unconditional effects
Weaker assumptions are more likely to be satisfied in practice
Good bc we aspire to explain optimization in all GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Proof Of Concept
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
fitness(x) simN (+025 1) if xi1 oplus xi2 oplus xi3 oplus xi4 = 1N (minus025 1) otherwise
0 Fitness
Pro
babi
lity
minus025 +025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
darr darr darr darr10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
uarr uarr uarr uarrEffective Loci
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Naive approach
Visit loci in groups of four
Check for differentiation inthe multivariate marginalfitness values
Time complexity is Ω(`4)
xi1 xi2 xi3 xi4
ExpectedMarginalFitness
0 0 0 0 minus0250 0 0 1 +0250 0 1 0 +0250 0 1 1 minus0250 1 0 0 +0250 1 0 1 minus0250 1 1 0 minus0250 1 1 1 +0251 0 0 0 +0251 0 0 1 minus0251 0 1 0 minus0251 0 1 1 +0251 1 0 0 minus0251 1 0 1 +0251 1 1 0 +0251 1 1 1 minus025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Visiting loci in groups ofthree or less wonrsquot work
xj
ExpectedMarginalFitness
0 001 00
xj1 xj2
ExpectedMarginalFitness
0 0 000 1 001 0 001 1 00
xj1 xj2 xj3
ExpectedMarginalFitness
0 0 0 000 0 1 000 1 0 000 1 1 001 0 0 001 0 1 001 1 0 001 1 1 00
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=200
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity200AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Symmetry Analytic Conclusion
Expected generations for the red dots to diverge constant wrt
Location of the red dots
Number of blue dots
Dynamics of a blue dot invariant to
Location of the red dots
Number of blue dots
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=1000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity1000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=10000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity10000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
Staircase function descriptor (m n δ σ ` LV )
L =
3 87 middot middot middot 93
42 58 middot middot middot 72
67 73 middot middot middot 81
︸ ︷︷ ︸
n
m V =
1 0 middot middot middot 11 0 middot middot middot 0
0 1 middot middot middot 1
︸ ︷︷ ︸
n
m
Staircase function is a stochastic function f 0 1` rarr R
0 δ 2δ (mminus1)δ mδ
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
δ = 02 σ = 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredInOrder__ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
trials=100 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
35 25 37 4631 5 32 2021 33 50 4240 27 30 312 14 48 2
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
B
Watch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
0 0 1 01 1 0 01 1 0 11 1 0 10 0 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=randommpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Prediction amp Validation
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
crosstype=2Mut=003Animmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
ctype=2StaircaseClampingAnimmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60000
100
200
300
400
500
600
700
800
900
Generation
Un
mu
tate
d lo
ci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
100
200
300
400
500
600
700
800
900
1000
Generation
Unm
uta
ted loci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Conclusion
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 22
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Variation in GAs
X1 X2 X3 X4 X5 X6 X7 X8 X9 X`
Y1 Y2 Y3 Y4 Y5 Y6 Y7 Y8 Y9 Y`
UX X1 X` are independent
Absence of positional bias (Eshelman et al 1989)
Attributes can be arbitrarily permuted
Suppose Y1 Y` independent and independent of `
If locus i immaterial to fitness can be spliced out
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
The Hyperclimbing Heuristic
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing A Stochastic Search Heuristic
The Setting
0 1` frarr R
f may be stochastic
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing A Stochastic Search Heuristic
The Setting
0 1` frarr R
f may be stochastic
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
The Effect of a Schema Partition
Effect of JIK Variance of average fitness of schemata in JIK
If I sub I prime then Effect(I) le Effect(I prime)
JIK
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
The Effect of a Schema Partition
Effect of JIK Variance of average fitness of schemata in JIK
If I sub I prime then Effect(I) le Effect(I prime)
JIK JI primeK
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption is Weak
Staggered coarse conditional effects assumption is weakWeaker than fitness distribution assumption in the BBH
BBH assumes unstaggered coarse unconditional effects
Weaker assumptions are more likely to be satisfied in practice
Good bc we aspire to explain optimization in all GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption is Weak
Staggered coarse conditional effects assumption is weakWeaker than fitness distribution assumption in the BBH
BBH assumes unstaggered coarse unconditional effects
Weaker assumptions are more likely to be satisfied in practice
Good bc we aspire to explain optimization in all GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Proof Of Concept
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
fitness(x) simN (+025 1) if xi1 oplus xi2 oplus xi3 oplus xi4 = 1N (minus025 1) otherwise
0 Fitness
Pro
babi
lity
minus025 +025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
darr darr darr darr10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
uarr uarr uarr uarrEffective Loci
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Naive approach
Visit loci in groups of four
Check for differentiation inthe multivariate marginalfitness values
Time complexity is Ω(`4)
xi1 xi2 xi3 xi4
ExpectedMarginalFitness
0 0 0 0 minus0250 0 0 1 +0250 0 1 0 +0250 0 1 1 minus0250 1 0 0 +0250 1 0 1 minus0250 1 1 0 minus0250 1 1 1 +0251 0 0 0 +0251 0 0 1 minus0251 0 1 0 minus0251 0 1 1 +0251 1 0 0 minus0251 1 0 1 +0251 1 1 0 +0251 1 1 1 minus025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Visiting loci in groups ofthree or less wonrsquot work
xj
ExpectedMarginalFitness
0 001 00
xj1 xj2
ExpectedMarginalFitness
0 0 000 1 001 0 001 1 00
xj1 xj2 xj3
ExpectedMarginalFitness
0 0 0 000 0 1 000 1 0 000 1 1 001 0 0 001 0 1 001 1 0 001 1 1 00
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=200
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity200AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Symmetry Analytic Conclusion
Expected generations for the red dots to diverge constant wrt
Location of the red dots
Number of blue dots
Dynamics of a blue dot invariant to
Location of the red dots
Number of blue dots
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=1000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity1000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=10000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity10000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
Staircase function descriptor (m n δ σ ` LV )
L =
3 87 middot middot middot 93
42 58 middot middot middot 72
67 73 middot middot middot 81
︸ ︷︷ ︸
n
m V =
1 0 middot middot middot 11 0 middot middot middot 0
0 1 middot middot middot 1
︸ ︷︷ ︸
n
m
Staircase function is a stochastic function f 0 1` rarr R
0 δ 2δ (mminus1)δ mδ
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
δ = 02 σ = 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredInOrder__ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
trials=100 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
35 25 37 4631 5 32 2021 33 50 4240 27 30 312 14 48 2
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
B
Watch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
0 0 1 01 1 0 01 1 0 11 1 0 10 0 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=randommpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Prediction amp Validation
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
crosstype=2Mut=003Animmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
ctype=2StaircaseClampingAnimmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60000
100
200
300
400
500
600
700
800
900
Generation
Un
mu
tate
d lo
ci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
100
200
300
400
500
600
700
800
900
1000
Generation
Unm
uta
ted loci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Conclusion
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 23
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
The Hyperclimbing Heuristic
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing A Stochastic Search Heuristic
The Setting
0 1` frarr R
f may be stochastic
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing A Stochastic Search Heuristic
The Setting
0 1` frarr R
f may be stochastic
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
The Effect of a Schema Partition
Effect of JIK Variance of average fitness of schemata in JIK
If I sub I prime then Effect(I) le Effect(I prime)
JIK
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
The Effect of a Schema Partition
Effect of JIK Variance of average fitness of schemata in JIK
If I sub I prime then Effect(I) le Effect(I prime)
JIK JI primeK
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption is Weak
Staggered coarse conditional effects assumption is weakWeaker than fitness distribution assumption in the BBH
BBH assumes unstaggered coarse unconditional effects
Weaker assumptions are more likely to be satisfied in practice
Good bc we aspire to explain optimization in all GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption is Weak
Staggered coarse conditional effects assumption is weakWeaker than fitness distribution assumption in the BBH
BBH assumes unstaggered coarse unconditional effects
Weaker assumptions are more likely to be satisfied in practice
Good bc we aspire to explain optimization in all GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Proof Of Concept
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
fitness(x) simN (+025 1) if xi1 oplus xi2 oplus xi3 oplus xi4 = 1N (minus025 1) otherwise
0 Fitness
Pro
babi
lity
minus025 +025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
darr darr darr darr10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
uarr uarr uarr uarrEffective Loci
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Naive approach
Visit loci in groups of four
Check for differentiation inthe multivariate marginalfitness values
Time complexity is Ω(`4)
xi1 xi2 xi3 xi4
ExpectedMarginalFitness
0 0 0 0 minus0250 0 0 1 +0250 0 1 0 +0250 0 1 1 minus0250 1 0 0 +0250 1 0 1 minus0250 1 1 0 minus0250 1 1 1 +0251 0 0 0 +0251 0 0 1 minus0251 0 1 0 minus0251 0 1 1 +0251 1 0 0 minus0251 1 0 1 +0251 1 1 0 +0251 1 1 1 minus025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Visiting loci in groups ofthree or less wonrsquot work
xj
ExpectedMarginalFitness
0 001 00
xj1 xj2
ExpectedMarginalFitness
0 0 000 1 001 0 001 1 00
xj1 xj2 xj3
ExpectedMarginalFitness
0 0 0 000 0 1 000 1 0 000 1 1 001 0 0 001 0 1 001 1 0 001 1 1 00
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=200
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity200AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Symmetry Analytic Conclusion
Expected generations for the red dots to diverge constant wrt
Location of the red dots
Number of blue dots
Dynamics of a blue dot invariant to
Location of the red dots
Number of blue dots
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=1000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity1000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=10000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity10000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
Staircase function descriptor (m n δ σ ` LV )
L =
3 87 middot middot middot 93
42 58 middot middot middot 72
67 73 middot middot middot 81
︸ ︷︷ ︸
n
m V =
1 0 middot middot middot 11 0 middot middot middot 0
0 1 middot middot middot 1
︸ ︷︷ ︸
n
m
Staircase function is a stochastic function f 0 1` rarr R
0 δ 2δ (mminus1)δ mδ
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
δ = 02 σ = 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredInOrder__ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
trials=100 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
35 25 37 4631 5 32 2021 33 50 4240 27 30 312 14 48 2
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
B
Watch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
0 0 1 01 1 0 01 1 0 11 1 0 10 0 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=randommpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Prediction amp Validation
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
crosstype=2Mut=003Animmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
ctype=2StaircaseClampingAnimmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60000
100
200
300
400
500
600
700
800
900
Generation
Un
mu
tate
d lo
ci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
100
200
300
400
500
600
700
800
900
1000
Generation
Unm
uta
ted loci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Conclusion
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 24
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing A Stochastic Search Heuristic
The Setting
0 1` frarr R
f may be stochastic
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing A Stochastic Search Heuristic
The Setting
0 1` frarr R
f may be stochastic
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
The Effect of a Schema Partition
Effect of JIK Variance of average fitness of schemata in JIK
If I sub I prime then Effect(I) le Effect(I prime)
JIK
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
The Effect of a Schema Partition
Effect of JIK Variance of average fitness of schemata in JIK
If I sub I prime then Effect(I) le Effect(I prime)
JIK JI primeK
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption is Weak
Staggered coarse conditional effects assumption is weakWeaker than fitness distribution assumption in the BBH
BBH assumes unstaggered coarse unconditional effects
Weaker assumptions are more likely to be satisfied in practice
Good bc we aspire to explain optimization in all GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption is Weak
Staggered coarse conditional effects assumption is weakWeaker than fitness distribution assumption in the BBH
BBH assumes unstaggered coarse unconditional effects
Weaker assumptions are more likely to be satisfied in practice
Good bc we aspire to explain optimization in all GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Proof Of Concept
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
fitness(x) simN (+025 1) if xi1 oplus xi2 oplus xi3 oplus xi4 = 1N (minus025 1) otherwise
0 Fitness
Pro
babi
lity
minus025 +025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
darr darr darr darr10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
uarr uarr uarr uarrEffective Loci
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Naive approach
Visit loci in groups of four
Check for differentiation inthe multivariate marginalfitness values
Time complexity is Ω(`4)
xi1 xi2 xi3 xi4
ExpectedMarginalFitness
0 0 0 0 minus0250 0 0 1 +0250 0 1 0 +0250 0 1 1 minus0250 1 0 0 +0250 1 0 1 minus0250 1 1 0 minus0250 1 1 1 +0251 0 0 0 +0251 0 0 1 minus0251 0 1 0 minus0251 0 1 1 +0251 1 0 0 minus0251 1 0 1 +0251 1 1 0 +0251 1 1 1 minus025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Visiting loci in groups ofthree or less wonrsquot work
xj
ExpectedMarginalFitness
0 001 00
xj1 xj2
ExpectedMarginalFitness
0 0 000 1 001 0 001 1 00
xj1 xj2 xj3
ExpectedMarginalFitness
0 0 0 000 0 1 000 1 0 000 1 1 001 0 0 001 0 1 001 1 0 001 1 1 00
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=200
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity200AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Symmetry Analytic Conclusion
Expected generations for the red dots to diverge constant wrt
Location of the red dots
Number of blue dots
Dynamics of a blue dot invariant to
Location of the red dots
Number of blue dots
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=1000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity1000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=10000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity10000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
Staircase function descriptor (m n δ σ ` LV )
L =
3 87 middot middot middot 93
42 58 middot middot middot 72
67 73 middot middot middot 81
︸ ︷︷ ︸
n
m V =
1 0 middot middot middot 11 0 middot middot middot 0
0 1 middot middot middot 1
︸ ︷︷ ︸
n
m
Staircase function is a stochastic function f 0 1` rarr R
0 δ 2δ (mminus1)δ mδ
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
δ = 02 σ = 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredInOrder__ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
trials=100 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
35 25 37 4631 5 32 2021 33 50 4240 27 30 312 14 48 2
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
B
Watch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
0 0 1 01 1 0 01 1 0 11 1 0 10 0 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=randommpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Prediction amp Validation
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
crosstype=2Mut=003Animmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
ctype=2StaircaseClampingAnimmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60000
100
200
300
400
500
600
700
800
900
Generation
Un
mu
tate
d lo
ci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
100
200
300
400
500
600
700
800
900
1000
Generation
Unm
uta
ted loci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Conclusion
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 25
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing A Stochastic Search Heuristic
The Setting
0 1` frarr R
f may be stochastic
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
The Effect of a Schema Partition
Effect of JIK Variance of average fitness of schemata in JIK
If I sub I prime then Effect(I) le Effect(I prime)
JIK
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
The Effect of a Schema Partition
Effect of JIK Variance of average fitness of schemata in JIK
If I sub I prime then Effect(I) le Effect(I prime)
JIK JI primeK
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption is Weak
Staggered coarse conditional effects assumption is weakWeaker than fitness distribution assumption in the BBH
BBH assumes unstaggered coarse unconditional effects
Weaker assumptions are more likely to be satisfied in practice
Good bc we aspire to explain optimization in all GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption is Weak
Staggered coarse conditional effects assumption is weakWeaker than fitness distribution assumption in the BBH
BBH assumes unstaggered coarse unconditional effects
Weaker assumptions are more likely to be satisfied in practice
Good bc we aspire to explain optimization in all GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Proof Of Concept
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
fitness(x) simN (+025 1) if xi1 oplus xi2 oplus xi3 oplus xi4 = 1N (minus025 1) otherwise
0 Fitness
Pro
babi
lity
minus025 +025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
darr darr darr darr10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
uarr uarr uarr uarrEffective Loci
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Naive approach
Visit loci in groups of four
Check for differentiation inthe multivariate marginalfitness values
Time complexity is Ω(`4)
xi1 xi2 xi3 xi4
ExpectedMarginalFitness
0 0 0 0 minus0250 0 0 1 +0250 0 1 0 +0250 0 1 1 minus0250 1 0 0 +0250 1 0 1 minus0250 1 1 0 minus0250 1 1 1 +0251 0 0 0 +0251 0 0 1 minus0251 0 1 0 minus0251 0 1 1 +0251 1 0 0 minus0251 1 0 1 +0251 1 1 0 +0251 1 1 1 minus025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Visiting loci in groups ofthree or less wonrsquot work
xj
ExpectedMarginalFitness
0 001 00
xj1 xj2
ExpectedMarginalFitness
0 0 000 1 001 0 001 1 00
xj1 xj2 xj3
ExpectedMarginalFitness
0 0 0 000 0 1 000 1 0 000 1 1 001 0 0 001 0 1 001 1 0 001 1 1 00
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=200
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity200AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Symmetry Analytic Conclusion
Expected generations for the red dots to diverge constant wrt
Location of the red dots
Number of blue dots
Dynamics of a blue dot invariant to
Location of the red dots
Number of blue dots
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=1000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity1000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=10000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity10000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
Staircase function descriptor (m n δ σ ` LV )
L =
3 87 middot middot middot 93
42 58 middot middot middot 72
67 73 middot middot middot 81
︸ ︷︷ ︸
n
m V =
1 0 middot middot middot 11 0 middot middot middot 0
0 1 middot middot middot 1
︸ ︷︷ ︸
n
m
Staircase function is a stochastic function f 0 1` rarr R
0 δ 2δ (mminus1)δ mδ
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
δ = 02 σ = 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredInOrder__ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
trials=100 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
35 25 37 4631 5 32 2021 33 50 4240 27 30 312 14 48 2
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
B
Watch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
0 0 1 01 1 0 01 1 0 11 1 0 10 0 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=randommpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Prediction amp Validation
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
crosstype=2Mut=003Animmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
ctype=2StaircaseClampingAnimmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60000
100
200
300
400
500
600
700
800
900
Generation
Un
mu
tate
d lo
ci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
100
200
300
400
500
600
700
800
900
1000
Generation
Unm
uta
ted loci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Conclusion
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 26
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
The Effect of a Schema Partition
Effect of JIK Variance of average fitness of schemata in JIK
If I sub I prime then Effect(I) le Effect(I prime)
JIK
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
The Effect of a Schema Partition
Effect of JIK Variance of average fitness of schemata in JIK
If I sub I prime then Effect(I) le Effect(I prime)
JIK JI primeK
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption is Weak
Staggered coarse conditional effects assumption is weakWeaker than fitness distribution assumption in the BBH
BBH assumes unstaggered coarse unconditional effects
Weaker assumptions are more likely to be satisfied in practice
Good bc we aspire to explain optimization in all GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption is Weak
Staggered coarse conditional effects assumption is weakWeaker than fitness distribution assumption in the BBH
BBH assumes unstaggered coarse unconditional effects
Weaker assumptions are more likely to be satisfied in practice
Good bc we aspire to explain optimization in all GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Proof Of Concept
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
fitness(x) simN (+025 1) if xi1 oplus xi2 oplus xi3 oplus xi4 = 1N (minus025 1) otherwise
0 Fitness
Pro
babi
lity
minus025 +025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
darr darr darr darr10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
uarr uarr uarr uarrEffective Loci
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Naive approach
Visit loci in groups of four
Check for differentiation inthe multivariate marginalfitness values
Time complexity is Ω(`4)
xi1 xi2 xi3 xi4
ExpectedMarginalFitness
0 0 0 0 minus0250 0 0 1 +0250 0 1 0 +0250 0 1 1 minus0250 1 0 0 +0250 1 0 1 minus0250 1 1 0 minus0250 1 1 1 +0251 0 0 0 +0251 0 0 1 minus0251 0 1 0 minus0251 0 1 1 +0251 1 0 0 minus0251 1 0 1 +0251 1 1 0 +0251 1 1 1 minus025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Visiting loci in groups ofthree or less wonrsquot work
xj
ExpectedMarginalFitness
0 001 00
xj1 xj2
ExpectedMarginalFitness
0 0 000 1 001 0 001 1 00
xj1 xj2 xj3
ExpectedMarginalFitness
0 0 0 000 0 1 000 1 0 000 1 1 001 0 0 001 0 1 001 1 0 001 1 1 00
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=200
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity200AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Symmetry Analytic Conclusion
Expected generations for the red dots to diverge constant wrt
Location of the red dots
Number of blue dots
Dynamics of a blue dot invariant to
Location of the red dots
Number of blue dots
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=1000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity1000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=10000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity10000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
Staircase function descriptor (m n δ σ ` LV )
L =
3 87 middot middot middot 93
42 58 middot middot middot 72
67 73 middot middot middot 81
︸ ︷︷ ︸
n
m V =
1 0 middot middot middot 11 0 middot middot middot 0
0 1 middot middot middot 1
︸ ︷︷ ︸
n
m
Staircase function is a stochastic function f 0 1` rarr R
0 δ 2δ (mminus1)δ mδ
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
δ = 02 σ = 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredInOrder__ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
trials=100 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
35 25 37 4631 5 32 2021 33 50 4240 27 30 312 14 48 2
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
B
Watch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
0 0 1 01 1 0 01 1 0 11 1 0 10 0 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=randommpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Prediction amp Validation
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
crosstype=2Mut=003Animmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
ctype=2StaircaseClampingAnimmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60000
100
200
300
400
500
600
700
800
900
Generation
Un
mu
tate
d lo
ci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
100
200
300
400
500
600
700
800
900
1000
Generation
Unm
uta
ted loci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Conclusion
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 27
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
The Effect of a Schema Partition
Effect of JIK Variance of average fitness of schemata in JIK
If I sub I prime then Effect(I) le Effect(I prime)
JIK
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
The Effect of a Schema Partition
Effect of JIK Variance of average fitness of schemata in JIK
If I sub I prime then Effect(I) le Effect(I prime)
JIK JI primeK
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption is Weak
Staggered coarse conditional effects assumption is weakWeaker than fitness distribution assumption in the BBH
BBH assumes unstaggered coarse unconditional effects
Weaker assumptions are more likely to be satisfied in practice
Good bc we aspire to explain optimization in all GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption is Weak
Staggered coarse conditional effects assumption is weakWeaker than fitness distribution assumption in the BBH
BBH assumes unstaggered coarse unconditional effects
Weaker assumptions are more likely to be satisfied in practice
Good bc we aspire to explain optimization in all GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Proof Of Concept
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
fitness(x) simN (+025 1) if xi1 oplus xi2 oplus xi3 oplus xi4 = 1N (minus025 1) otherwise
0 Fitness
Pro
babi
lity
minus025 +025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
darr darr darr darr10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
uarr uarr uarr uarrEffective Loci
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Naive approach
Visit loci in groups of four
Check for differentiation inthe multivariate marginalfitness values
Time complexity is Ω(`4)
xi1 xi2 xi3 xi4
ExpectedMarginalFitness
0 0 0 0 minus0250 0 0 1 +0250 0 1 0 +0250 0 1 1 minus0250 1 0 0 +0250 1 0 1 minus0250 1 1 0 minus0250 1 1 1 +0251 0 0 0 +0251 0 0 1 minus0251 0 1 0 minus0251 0 1 1 +0251 1 0 0 minus0251 1 0 1 +0251 1 1 0 +0251 1 1 1 minus025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Visiting loci in groups ofthree or less wonrsquot work
xj
ExpectedMarginalFitness
0 001 00
xj1 xj2
ExpectedMarginalFitness
0 0 000 1 001 0 001 1 00
xj1 xj2 xj3
ExpectedMarginalFitness
0 0 0 000 0 1 000 1 0 000 1 1 001 0 0 001 0 1 001 1 0 001 1 1 00
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=200
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity200AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Symmetry Analytic Conclusion
Expected generations for the red dots to diverge constant wrt
Location of the red dots
Number of blue dots
Dynamics of a blue dot invariant to
Location of the red dots
Number of blue dots
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=1000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity1000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=10000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity10000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
Staircase function descriptor (m n δ σ ` LV )
L =
3 87 middot middot middot 93
42 58 middot middot middot 72
67 73 middot middot middot 81
︸ ︷︷ ︸
n
m V =
1 0 middot middot middot 11 0 middot middot middot 0
0 1 middot middot middot 1
︸ ︷︷ ︸
n
m
Staircase function is a stochastic function f 0 1` rarr R
0 δ 2δ (mminus1)δ mδ
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
δ = 02 σ = 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredInOrder__ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
trials=100 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
35 25 37 4631 5 32 2021 33 50 4240 27 30 312 14 48 2
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
B
Watch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
0 0 1 01 1 0 01 1 0 11 1 0 10 0 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=randommpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Prediction amp Validation
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
crosstype=2Mut=003Animmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
ctype=2StaircaseClampingAnimmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60000
100
200
300
400
500
600
700
800
900
Generation
Un
mu
tate
d lo
ci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
100
200
300
400
500
600
700
800
900
1000
Generation
Unm
uta
ted loci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Conclusion
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 28
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
The Effect of a Schema Partition
Effect of JIK Variance of average fitness of schemata in JIK
If I sub I prime then Effect(I) le Effect(I prime)
JIK
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
The Effect of a Schema Partition
Effect of JIK Variance of average fitness of schemata in JIK
If I sub I prime then Effect(I) le Effect(I prime)
JIK JI primeK
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption is Weak
Staggered coarse conditional effects assumption is weakWeaker than fitness distribution assumption in the BBH
BBH assumes unstaggered coarse unconditional effects
Weaker assumptions are more likely to be satisfied in practice
Good bc we aspire to explain optimization in all GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption is Weak
Staggered coarse conditional effects assumption is weakWeaker than fitness distribution assumption in the BBH
BBH assumes unstaggered coarse unconditional effects
Weaker assumptions are more likely to be satisfied in practice
Good bc we aspire to explain optimization in all GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Proof Of Concept
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
fitness(x) simN (+025 1) if xi1 oplus xi2 oplus xi3 oplus xi4 = 1N (minus025 1) otherwise
0 Fitness
Pro
babi
lity
minus025 +025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
darr darr darr darr10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
uarr uarr uarr uarrEffective Loci
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Naive approach
Visit loci in groups of four
Check for differentiation inthe multivariate marginalfitness values
Time complexity is Ω(`4)
xi1 xi2 xi3 xi4
ExpectedMarginalFitness
0 0 0 0 minus0250 0 0 1 +0250 0 1 0 +0250 0 1 1 minus0250 1 0 0 +0250 1 0 1 minus0250 1 1 0 minus0250 1 1 1 +0251 0 0 0 +0251 0 0 1 minus0251 0 1 0 minus0251 0 1 1 +0251 1 0 0 minus0251 1 0 1 +0251 1 1 0 +0251 1 1 1 minus025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Visiting loci in groups ofthree or less wonrsquot work
xj
ExpectedMarginalFitness
0 001 00
xj1 xj2
ExpectedMarginalFitness
0 0 000 1 001 0 001 1 00
xj1 xj2 xj3
ExpectedMarginalFitness
0 0 0 000 0 1 000 1 0 000 1 1 001 0 0 001 0 1 001 1 0 001 1 1 00
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=200
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity200AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Symmetry Analytic Conclusion
Expected generations for the red dots to diverge constant wrt
Location of the red dots
Number of blue dots
Dynamics of a blue dot invariant to
Location of the red dots
Number of blue dots
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=1000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity1000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=10000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity10000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
Staircase function descriptor (m n δ σ ` LV )
L =
3 87 middot middot middot 93
42 58 middot middot middot 72
67 73 middot middot middot 81
︸ ︷︷ ︸
n
m V =
1 0 middot middot middot 11 0 middot middot middot 0
0 1 middot middot middot 1
︸ ︷︷ ︸
n
m
Staircase function is a stochastic function f 0 1` rarr R
0 δ 2δ (mminus1)δ mδ
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
δ = 02 σ = 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredInOrder__ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
trials=100 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
35 25 37 4631 5 32 2021 33 50 4240 27 30 312 14 48 2
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
B
Watch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
0 0 1 01 1 0 01 1 0 11 1 0 10 0 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=randommpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Prediction amp Validation
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
crosstype=2Mut=003Animmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
ctype=2StaircaseClampingAnimmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60000
100
200
300
400
500
600
700
800
900
Generation
Un
mu
tate
d lo
ci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
100
200
300
400
500
600
700
800
900
1000
Generation
Unm
uta
ted loci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Conclusion
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 29
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Schema Partitions and Schemata
Given index set I sub 1 ` st |I| = k
I partitions the search space into 2k schemata
Schema partition denoted by JIK
Eg for I = 3 7
A3 = 0 A7 = 0
A3 = 0 A7 = 1
A3 = 1 A7 = 1A3 = 1 A7 = 0
Coarseness of the partitiondetermined by k (the order)(`k
)schema partitions of
order k
For fixed k(`k
)isin Ω(`k)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
The Effect of a Schema Partition
Effect of JIK Variance of average fitness of schemata in JIK
If I sub I prime then Effect(I) le Effect(I prime)
JIK
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
The Effect of a Schema Partition
Effect of JIK Variance of average fitness of schemata in JIK
If I sub I prime then Effect(I) le Effect(I prime)
JIK JI primeK
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption is Weak
Staggered coarse conditional effects assumption is weakWeaker than fitness distribution assumption in the BBH
BBH assumes unstaggered coarse unconditional effects
Weaker assumptions are more likely to be satisfied in practice
Good bc we aspire to explain optimization in all GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption is Weak
Staggered coarse conditional effects assumption is weakWeaker than fitness distribution assumption in the BBH
BBH assumes unstaggered coarse unconditional effects
Weaker assumptions are more likely to be satisfied in practice
Good bc we aspire to explain optimization in all GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Proof Of Concept
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
fitness(x) simN (+025 1) if xi1 oplus xi2 oplus xi3 oplus xi4 = 1N (minus025 1) otherwise
0 Fitness
Pro
babi
lity
minus025 +025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
darr darr darr darr10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
uarr uarr uarr uarrEffective Loci
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Naive approach
Visit loci in groups of four
Check for differentiation inthe multivariate marginalfitness values
Time complexity is Ω(`4)
xi1 xi2 xi3 xi4
ExpectedMarginalFitness
0 0 0 0 minus0250 0 0 1 +0250 0 1 0 +0250 0 1 1 minus0250 1 0 0 +0250 1 0 1 minus0250 1 1 0 minus0250 1 1 1 +0251 0 0 0 +0251 0 0 1 minus0251 0 1 0 minus0251 0 1 1 +0251 1 0 0 minus0251 1 0 1 +0251 1 1 0 +0251 1 1 1 minus025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Visiting loci in groups ofthree or less wonrsquot work
xj
ExpectedMarginalFitness
0 001 00
xj1 xj2
ExpectedMarginalFitness
0 0 000 1 001 0 001 1 00
xj1 xj2 xj3
ExpectedMarginalFitness
0 0 0 000 0 1 000 1 0 000 1 1 001 0 0 001 0 1 001 1 0 001 1 1 00
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=200
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity200AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Symmetry Analytic Conclusion
Expected generations for the red dots to diverge constant wrt
Location of the red dots
Number of blue dots
Dynamics of a blue dot invariant to
Location of the red dots
Number of blue dots
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=1000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity1000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=10000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity10000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
Staircase function descriptor (m n δ σ ` LV )
L =
3 87 middot middot middot 93
42 58 middot middot middot 72
67 73 middot middot middot 81
︸ ︷︷ ︸
n
m V =
1 0 middot middot middot 11 0 middot middot middot 0
0 1 middot middot middot 1
︸ ︷︷ ︸
n
m
Staircase function is a stochastic function f 0 1` rarr R
0 δ 2δ (mminus1)δ mδ
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
δ = 02 σ = 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredInOrder__ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
trials=100 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
35 25 37 4631 5 32 2021 33 50 4240 27 30 312 14 48 2
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
B
Watch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
0 0 1 01 1 0 01 1 0 11 1 0 10 0 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=randommpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Prediction amp Validation
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
crosstype=2Mut=003Animmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
ctype=2StaircaseClampingAnimmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60000
100
200
300
400
500
600
700
800
900
Generation
Un
mu
tate
d lo
ci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
100
200
300
400
500
600
700
800
900
1000
Generation
Unm
uta
ted loci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Conclusion
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 30
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
The Effect of a Schema Partition
Effect of JIK Variance of average fitness of schemata in JIK
If I sub I prime then Effect(I) le Effect(I prime)
JIK
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
The Effect of a Schema Partition
Effect of JIK Variance of average fitness of schemata in JIK
If I sub I prime then Effect(I) le Effect(I prime)
JIK JI primeK
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption is Weak
Staggered coarse conditional effects assumption is weakWeaker than fitness distribution assumption in the BBH
BBH assumes unstaggered coarse unconditional effects
Weaker assumptions are more likely to be satisfied in practice
Good bc we aspire to explain optimization in all GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption is Weak
Staggered coarse conditional effects assumption is weakWeaker than fitness distribution assumption in the BBH
BBH assumes unstaggered coarse unconditional effects
Weaker assumptions are more likely to be satisfied in practice
Good bc we aspire to explain optimization in all GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Proof Of Concept
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
fitness(x) simN (+025 1) if xi1 oplus xi2 oplus xi3 oplus xi4 = 1N (minus025 1) otherwise
0 Fitness
Pro
babi
lity
minus025 +025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
darr darr darr darr10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
uarr uarr uarr uarrEffective Loci
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Naive approach
Visit loci in groups of four
Check for differentiation inthe multivariate marginalfitness values
Time complexity is Ω(`4)
xi1 xi2 xi3 xi4
ExpectedMarginalFitness
0 0 0 0 minus0250 0 0 1 +0250 0 1 0 +0250 0 1 1 minus0250 1 0 0 +0250 1 0 1 minus0250 1 1 0 minus0250 1 1 1 +0251 0 0 0 +0251 0 0 1 minus0251 0 1 0 minus0251 0 1 1 +0251 1 0 0 minus0251 1 0 1 +0251 1 1 0 +0251 1 1 1 minus025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Visiting loci in groups ofthree or less wonrsquot work
xj
ExpectedMarginalFitness
0 001 00
xj1 xj2
ExpectedMarginalFitness
0 0 000 1 001 0 001 1 00
xj1 xj2 xj3
ExpectedMarginalFitness
0 0 0 000 0 1 000 1 0 000 1 1 001 0 0 001 0 1 001 1 0 001 1 1 00
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=200
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity200AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Symmetry Analytic Conclusion
Expected generations for the red dots to diverge constant wrt
Location of the red dots
Number of blue dots
Dynamics of a blue dot invariant to
Location of the red dots
Number of blue dots
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=1000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity1000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=10000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity10000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
Staircase function descriptor (m n δ σ ` LV )
L =
3 87 middot middot middot 93
42 58 middot middot middot 72
67 73 middot middot middot 81
︸ ︷︷ ︸
n
m V =
1 0 middot middot middot 11 0 middot middot middot 0
0 1 middot middot middot 1
︸ ︷︷ ︸
n
m
Staircase function is a stochastic function f 0 1` rarr R
0 δ 2δ (mminus1)δ mδ
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
δ = 02 σ = 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredInOrder__ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
trials=100 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
35 25 37 4631 5 32 2021 33 50 4240 27 30 312 14 48 2
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
B
Watch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
0 0 1 01 1 0 01 1 0 11 1 0 10 0 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=randommpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Prediction amp Validation
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
crosstype=2Mut=003Animmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
ctype=2StaircaseClampingAnimmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60000
100
200
300
400
500
600
700
800
900
Generation
Un
mu
tate
d lo
ci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
100
200
300
400
500
600
700
800
900
1000
Generation
Unm
uta
ted loci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Conclusion
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 31
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
The Effect of a Schema Partition
Effect of JIK Variance of average fitness of schemata in JIK
If I sub I prime then Effect(I) le Effect(I prime)
JIK JI primeK
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption is Weak
Staggered coarse conditional effects assumption is weakWeaker than fitness distribution assumption in the BBH
BBH assumes unstaggered coarse unconditional effects
Weaker assumptions are more likely to be satisfied in practice
Good bc we aspire to explain optimization in all GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption is Weak
Staggered coarse conditional effects assumption is weakWeaker than fitness distribution assumption in the BBH
BBH assumes unstaggered coarse unconditional effects
Weaker assumptions are more likely to be satisfied in practice
Good bc we aspire to explain optimization in all GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Proof Of Concept
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
fitness(x) simN (+025 1) if xi1 oplus xi2 oplus xi3 oplus xi4 = 1N (minus025 1) otherwise
0 Fitness
Pro
babi
lity
minus025 +025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
darr darr darr darr10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
uarr uarr uarr uarrEffective Loci
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Naive approach
Visit loci in groups of four
Check for differentiation inthe multivariate marginalfitness values
Time complexity is Ω(`4)
xi1 xi2 xi3 xi4
ExpectedMarginalFitness
0 0 0 0 minus0250 0 0 1 +0250 0 1 0 +0250 0 1 1 minus0250 1 0 0 +0250 1 0 1 minus0250 1 1 0 minus0250 1 1 1 +0251 0 0 0 +0251 0 0 1 minus0251 0 1 0 minus0251 0 1 1 +0251 1 0 0 minus0251 1 0 1 +0251 1 1 0 +0251 1 1 1 minus025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Visiting loci in groups ofthree or less wonrsquot work
xj
ExpectedMarginalFitness
0 001 00
xj1 xj2
ExpectedMarginalFitness
0 0 000 1 001 0 001 1 00
xj1 xj2 xj3
ExpectedMarginalFitness
0 0 0 000 0 1 000 1 0 000 1 1 001 0 0 001 0 1 001 1 0 001 1 1 00
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=200
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity200AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Symmetry Analytic Conclusion
Expected generations for the red dots to diverge constant wrt
Location of the red dots
Number of blue dots
Dynamics of a blue dot invariant to
Location of the red dots
Number of blue dots
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=1000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity1000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=10000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity10000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
Staircase function descriptor (m n δ σ ` LV )
L =
3 87 middot middot middot 93
42 58 middot middot middot 72
67 73 middot middot middot 81
︸ ︷︷ ︸
n
m V =
1 0 middot middot middot 11 0 middot middot middot 0
0 1 middot middot middot 1
︸ ︷︷ ︸
n
m
Staircase function is a stochastic function f 0 1` rarr R
0 δ 2δ (mminus1)δ mδ
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
δ = 02 σ = 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredInOrder__ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
trials=100 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
35 25 37 4631 5 32 2021 33 50 4240 27 30 312 14 48 2
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
B
Watch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
0 0 1 01 1 0 01 1 0 11 1 0 10 0 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=randommpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Prediction amp Validation
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
crosstype=2Mut=003Animmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
ctype=2StaircaseClampingAnimmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60000
100
200
300
400
500
600
700
800
900
Generation
Un
mu
tate
d lo
ci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
100
200
300
400
500
600
700
800
900
1000
Generation
Unm
uta
ted loci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Conclusion
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 32
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption is Weak
Staggered coarse conditional effects assumption is weakWeaker than fitness distribution assumption in the BBH
BBH assumes unstaggered coarse unconditional effects
Weaker assumptions are more likely to be satisfied in practice
Good bc we aspire to explain optimization in all GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption is Weak
Staggered coarse conditional effects assumption is weakWeaker than fitness distribution assumption in the BBH
BBH assumes unstaggered coarse unconditional effects
Weaker assumptions are more likely to be satisfied in practice
Good bc we aspire to explain optimization in all GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Proof Of Concept
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
fitness(x) simN (+025 1) if xi1 oplus xi2 oplus xi3 oplus xi4 = 1N (minus025 1) otherwise
0 Fitness
Pro
babi
lity
minus025 +025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
darr darr darr darr10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
uarr uarr uarr uarrEffective Loci
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Naive approach
Visit loci in groups of four
Check for differentiation inthe multivariate marginalfitness values
Time complexity is Ω(`4)
xi1 xi2 xi3 xi4
ExpectedMarginalFitness
0 0 0 0 minus0250 0 0 1 +0250 0 1 0 +0250 0 1 1 minus0250 1 0 0 +0250 1 0 1 minus0250 1 1 0 minus0250 1 1 1 +0251 0 0 0 +0251 0 0 1 minus0251 0 1 0 minus0251 0 1 1 +0251 1 0 0 minus0251 1 0 1 +0251 1 1 0 +0251 1 1 1 minus025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Visiting loci in groups ofthree or less wonrsquot work
xj
ExpectedMarginalFitness
0 001 00
xj1 xj2
ExpectedMarginalFitness
0 0 000 1 001 0 001 1 00
xj1 xj2 xj3
ExpectedMarginalFitness
0 0 0 000 0 1 000 1 0 000 1 1 001 0 0 001 0 1 001 1 0 001 1 1 00
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=200
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity200AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Symmetry Analytic Conclusion
Expected generations for the red dots to diverge constant wrt
Location of the red dots
Number of blue dots
Dynamics of a blue dot invariant to
Location of the red dots
Number of blue dots
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=1000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity1000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=10000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity10000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
Staircase function descriptor (m n δ σ ` LV )
L =
3 87 middot middot middot 93
42 58 middot middot middot 72
67 73 middot middot middot 81
︸ ︷︷ ︸
n
m V =
1 0 middot middot middot 11 0 middot middot middot 0
0 1 middot middot middot 1
︸ ︷︷ ︸
n
m
Staircase function is a stochastic function f 0 1` rarr R
0 δ 2δ (mminus1)δ mδ
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
δ = 02 σ = 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredInOrder__ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
trials=100 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
35 25 37 4631 5 32 2021 33 50 4240 27 30 312 14 48 2
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
B
Watch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
0 0 1 01 1 0 01 1 0 11 1 0 10 0 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=randommpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Prediction amp Validation
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
crosstype=2Mut=003Animmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
ctype=2StaircaseClampingAnimmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60000
100
200
300
400
500
600
700
800
900
Generation
Un
mu
tate
d lo
ci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
100
200
300
400
500
600
700
800
900
1000
Generation
Unm
uta
ted loci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Conclusion
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 33
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption is Weak
Staggered coarse conditional effects assumption is weakWeaker than fitness distribution assumption in the BBH
BBH assumes unstaggered coarse unconditional effects
Weaker assumptions are more likely to be satisfied in practice
Good bc we aspire to explain optimization in all GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption is Weak
Staggered coarse conditional effects assumption is weakWeaker than fitness distribution assumption in the BBH
BBH assumes unstaggered coarse unconditional effects
Weaker assumptions are more likely to be satisfied in practice
Good bc we aspire to explain optimization in all GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Proof Of Concept
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
fitness(x) simN (+025 1) if xi1 oplus xi2 oplus xi3 oplus xi4 = 1N (minus025 1) otherwise
0 Fitness
Pro
babi
lity
minus025 +025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
darr darr darr darr10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
uarr uarr uarr uarrEffective Loci
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Naive approach
Visit loci in groups of four
Check for differentiation inthe multivariate marginalfitness values
Time complexity is Ω(`4)
xi1 xi2 xi3 xi4
ExpectedMarginalFitness
0 0 0 0 minus0250 0 0 1 +0250 0 1 0 +0250 0 1 1 minus0250 1 0 0 +0250 1 0 1 minus0250 1 1 0 minus0250 1 1 1 +0251 0 0 0 +0251 0 0 1 minus0251 0 1 0 minus0251 0 1 1 +0251 1 0 0 minus0251 1 0 1 +0251 1 1 0 +0251 1 1 1 minus025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Visiting loci in groups ofthree or less wonrsquot work
xj
ExpectedMarginalFitness
0 001 00
xj1 xj2
ExpectedMarginalFitness
0 0 000 1 001 0 001 1 00
xj1 xj2 xj3
ExpectedMarginalFitness
0 0 0 000 0 1 000 1 0 000 1 1 001 0 0 001 0 1 001 1 0 001 1 1 00
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=200
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity200AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Symmetry Analytic Conclusion
Expected generations for the red dots to diverge constant wrt
Location of the red dots
Number of blue dots
Dynamics of a blue dot invariant to
Location of the red dots
Number of blue dots
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=1000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity1000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=10000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity10000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
Staircase function descriptor (m n δ σ ` LV )
L =
3 87 middot middot middot 93
42 58 middot middot middot 72
67 73 middot middot middot 81
︸ ︷︷ ︸
n
m V =
1 0 middot middot middot 11 0 middot middot middot 0
0 1 middot middot middot 1
︸ ︷︷ ︸
n
m
Staircase function is a stochastic function f 0 1` rarr R
0 δ 2δ (mminus1)δ mδ
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
δ = 02 σ = 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredInOrder__ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
trials=100 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
35 25 37 4631 5 32 2021 33 50 4240 27 30 312 14 48 2
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
B
Watch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
0 0 1 01 1 0 01 1 0 11 1 0 10 0 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=randommpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Prediction amp Validation
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
crosstype=2Mut=003Animmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
ctype=2StaircaseClampingAnimmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60000
100
200
300
400
500
600
700
800
900
Generation
Un
mu
tate
d lo
ci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
100
200
300
400
500
600
700
800
900
1000
Generation
Unm
uta
ted loci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Conclusion
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 34
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption is Weak
Staggered coarse conditional effects assumption is weakWeaker than fitness distribution assumption in the BBH
BBH assumes unstaggered coarse unconditional effects
Weaker assumptions are more likely to be satisfied in practice
Good bc we aspire to explain optimization in all GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption is Weak
Staggered coarse conditional effects assumption is weakWeaker than fitness distribution assumption in the BBH
BBH assumes unstaggered coarse unconditional effects
Weaker assumptions are more likely to be satisfied in practice
Good bc we aspire to explain optimization in all GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Proof Of Concept
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
fitness(x) simN (+025 1) if xi1 oplus xi2 oplus xi3 oplus xi4 = 1N (minus025 1) otherwise
0 Fitness
Pro
babi
lity
minus025 +025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
darr darr darr darr10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
uarr uarr uarr uarrEffective Loci
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Naive approach
Visit loci in groups of four
Check for differentiation inthe multivariate marginalfitness values
Time complexity is Ω(`4)
xi1 xi2 xi3 xi4
ExpectedMarginalFitness
0 0 0 0 minus0250 0 0 1 +0250 0 1 0 +0250 0 1 1 minus0250 1 0 0 +0250 1 0 1 minus0250 1 1 0 minus0250 1 1 1 +0251 0 0 0 +0251 0 0 1 minus0251 0 1 0 minus0251 0 1 1 +0251 1 0 0 minus0251 1 0 1 +0251 1 1 0 +0251 1 1 1 minus025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Visiting loci in groups ofthree or less wonrsquot work
xj
ExpectedMarginalFitness
0 001 00
xj1 xj2
ExpectedMarginalFitness
0 0 000 1 001 0 001 1 00
xj1 xj2 xj3
ExpectedMarginalFitness
0 0 0 000 0 1 000 1 0 000 1 1 001 0 0 001 0 1 001 1 0 001 1 1 00
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=200
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity200AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Symmetry Analytic Conclusion
Expected generations for the red dots to diverge constant wrt
Location of the red dots
Number of blue dots
Dynamics of a blue dot invariant to
Location of the red dots
Number of blue dots
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=1000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity1000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=10000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity10000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
Staircase function descriptor (m n δ σ ` LV )
L =
3 87 middot middot middot 93
42 58 middot middot middot 72
67 73 middot middot middot 81
︸ ︷︷ ︸
n
m V =
1 0 middot middot middot 11 0 middot middot middot 0
0 1 middot middot middot 1
︸ ︷︷ ︸
n
m
Staircase function is a stochastic function f 0 1` rarr R
0 δ 2δ (mminus1)δ mδ
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
δ = 02 σ = 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredInOrder__ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
trials=100 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
35 25 37 4631 5 32 2021 33 50 4240 27 30 312 14 48 2
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
B
Watch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
0 0 1 01 1 0 01 1 0 11 1 0 10 0 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=randommpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Prediction amp Validation
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
crosstype=2Mut=003Animmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
ctype=2StaircaseClampingAnimmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60000
100
200
300
400
500
600
700
800
900
Generation
Un
mu
tate
d lo
ci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
100
200
300
400
500
600
700
800
900
1000
Generation
Unm
uta
ted loci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Conclusion
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 35
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Finds a coarse schema partitionwith a detectable effect
Limits future sampling to aschema with above averagefitness
Raises expected fitness of all future samples
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption is Weak
Staggered coarse conditional effects assumption is weakWeaker than fitness distribution assumption in the BBH
BBH assumes unstaggered coarse unconditional effects
Weaker assumptions are more likely to be satisfied in practice
Good bc we aspire to explain optimization in all GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption is Weak
Staggered coarse conditional effects assumption is weakWeaker than fitness distribution assumption in the BBH
BBH assumes unstaggered coarse unconditional effects
Weaker assumptions are more likely to be satisfied in practice
Good bc we aspire to explain optimization in all GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Proof Of Concept
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
fitness(x) simN (+025 1) if xi1 oplus xi2 oplus xi3 oplus xi4 = 1N (minus025 1) otherwise
0 Fitness
Pro
babi
lity
minus025 +025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
darr darr darr darr10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
uarr uarr uarr uarrEffective Loci
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Naive approach
Visit loci in groups of four
Check for differentiation inthe multivariate marginalfitness values
Time complexity is Ω(`4)
xi1 xi2 xi3 xi4
ExpectedMarginalFitness
0 0 0 0 minus0250 0 0 1 +0250 0 1 0 +0250 0 1 1 minus0250 1 0 0 +0250 1 0 1 minus0250 1 1 0 minus0250 1 1 1 +0251 0 0 0 +0251 0 0 1 minus0251 0 1 0 minus0251 0 1 1 +0251 1 0 0 minus0251 1 0 1 +0251 1 1 0 +0251 1 1 1 minus025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Visiting loci in groups ofthree or less wonrsquot work
xj
ExpectedMarginalFitness
0 001 00
xj1 xj2
ExpectedMarginalFitness
0 0 000 1 001 0 001 1 00
xj1 xj2 xj3
ExpectedMarginalFitness
0 0 0 000 0 1 000 1 0 000 1 1 001 0 0 001 0 1 001 1 0 001 1 1 00
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=200
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity200AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Symmetry Analytic Conclusion
Expected generations for the red dots to diverge constant wrt
Location of the red dots
Number of blue dots
Dynamics of a blue dot invariant to
Location of the red dots
Number of blue dots
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=1000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity1000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=10000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity10000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
Staircase function descriptor (m n δ σ ` LV )
L =
3 87 middot middot middot 93
42 58 middot middot middot 72
67 73 middot middot middot 81
︸ ︷︷ ︸
n
m V =
1 0 middot middot middot 11 0 middot middot middot 0
0 1 middot middot middot 1
︸ ︷︷ ︸
n
m
Staircase function is a stochastic function f 0 1` rarr R
0 δ 2δ (mminus1)δ mδ
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
δ = 02 σ = 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredInOrder__ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
trials=100 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
35 25 37 4631 5 32 2021 33 50 4240 27 30 312 14 48 2
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
B
Watch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
0 0 1 01 1 0 01 1 0 11 1 0 10 0 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=randommpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Prediction amp Validation
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
crosstype=2Mut=003Animmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
ctype=2StaircaseClampingAnimmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60000
100
200
300
400
500
600
700
800
900
Generation
Un
mu
tate
d lo
ci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
100
200
300
400
500
600
700
800
900
1000
Generation
Unm
uta
ted loci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Conclusion
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 36
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption is Weak
Staggered coarse conditional effects assumption is weakWeaker than fitness distribution assumption in the BBH
BBH assumes unstaggered coarse unconditional effects
Weaker assumptions are more likely to be satisfied in practice
Good bc we aspire to explain optimization in all GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption is Weak
Staggered coarse conditional effects assumption is weakWeaker than fitness distribution assumption in the BBH
BBH assumes unstaggered coarse unconditional effects
Weaker assumptions are more likely to be satisfied in practice
Good bc we aspire to explain optimization in all GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Proof Of Concept
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
fitness(x) simN (+025 1) if xi1 oplus xi2 oplus xi3 oplus xi4 = 1N (minus025 1) otherwise
0 Fitness
Pro
babi
lity
minus025 +025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
darr darr darr darr10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
uarr uarr uarr uarrEffective Loci
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Naive approach
Visit loci in groups of four
Check for differentiation inthe multivariate marginalfitness values
Time complexity is Ω(`4)
xi1 xi2 xi3 xi4
ExpectedMarginalFitness
0 0 0 0 minus0250 0 0 1 +0250 0 1 0 +0250 0 1 1 minus0250 1 0 0 +0250 1 0 1 minus0250 1 1 0 minus0250 1 1 1 +0251 0 0 0 +0251 0 0 1 minus0251 0 1 0 minus0251 0 1 1 +0251 1 0 0 minus0251 1 0 1 +0251 1 1 0 +0251 1 1 1 minus025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Visiting loci in groups ofthree or less wonrsquot work
xj
ExpectedMarginalFitness
0 001 00
xj1 xj2
ExpectedMarginalFitness
0 0 000 1 001 0 001 1 00
xj1 xj2 xj3
ExpectedMarginalFitness
0 0 0 000 0 1 000 1 0 000 1 1 001 0 0 001 0 1 001 1 0 001 1 1 00
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=200
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity200AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Symmetry Analytic Conclusion
Expected generations for the red dots to diverge constant wrt
Location of the red dots
Number of blue dots
Dynamics of a blue dot invariant to
Location of the red dots
Number of blue dots
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=1000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity1000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=10000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity10000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
Staircase function descriptor (m n δ σ ` LV )
L =
3 87 middot middot middot 93
42 58 middot middot middot 72
67 73 middot middot middot 81
︸ ︷︷ ︸
n
m V =
1 0 middot middot middot 11 0 middot middot middot 0
0 1 middot middot middot 1
︸ ︷︷ ︸
n
m
Staircase function is a stochastic function f 0 1` rarr R
0 δ 2δ (mminus1)δ mδ
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
δ = 02 σ = 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredInOrder__ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
trials=100 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
35 25 37 4631 5 32 2021 33 50 4240 27 30 312 14 48 2
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
B
Watch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
0 0 1 01 1 0 01 1 0 11 1 0 10 0 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=randommpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Prediction amp Validation
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
crosstype=2Mut=003Animmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
ctype=2StaircaseClampingAnimmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60000
100
200
300
400
500
600
700
800
900
Generation
Un
mu
tate
d lo
ci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
100
200
300
400
500
600
700
800
900
1000
Generation
Unm
uta
ted loci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Conclusion
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 37
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption is Weak
Staggered coarse conditional effects assumption is weakWeaker than fitness distribution assumption in the BBH
BBH assumes unstaggered coarse unconditional effects
Weaker assumptions are more likely to be satisfied in practice
Good bc we aspire to explain optimization in all GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption is Weak
Staggered coarse conditional effects assumption is weakWeaker than fitness distribution assumption in the BBH
BBH assumes unstaggered coarse unconditional effects
Weaker assumptions are more likely to be satisfied in practice
Good bc we aspire to explain optimization in all GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Proof Of Concept
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
fitness(x) simN (+025 1) if xi1 oplus xi2 oplus xi3 oplus xi4 = 1N (minus025 1) otherwise
0 Fitness
Pro
babi
lity
minus025 +025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
darr darr darr darr10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
uarr uarr uarr uarrEffective Loci
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Naive approach
Visit loci in groups of four
Check for differentiation inthe multivariate marginalfitness values
Time complexity is Ω(`4)
xi1 xi2 xi3 xi4
ExpectedMarginalFitness
0 0 0 0 minus0250 0 0 1 +0250 0 1 0 +0250 0 1 1 minus0250 1 0 0 +0250 1 0 1 minus0250 1 1 0 minus0250 1 1 1 +0251 0 0 0 +0251 0 0 1 minus0251 0 1 0 minus0251 0 1 1 +0251 1 0 0 minus0251 1 0 1 +0251 1 1 0 +0251 1 1 1 minus025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Visiting loci in groups ofthree or less wonrsquot work
xj
ExpectedMarginalFitness
0 001 00
xj1 xj2
ExpectedMarginalFitness
0 0 000 1 001 0 001 1 00
xj1 xj2 xj3
ExpectedMarginalFitness
0 0 0 000 0 1 000 1 0 000 1 1 001 0 0 001 0 1 001 1 0 001 1 1 00
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=200
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity200AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Symmetry Analytic Conclusion
Expected generations for the red dots to diverge constant wrt
Location of the red dots
Number of blue dots
Dynamics of a blue dot invariant to
Location of the red dots
Number of blue dots
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=1000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity1000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=10000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity10000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
Staircase function descriptor (m n δ σ ` LV )
L =
3 87 middot middot middot 93
42 58 middot middot middot 72
67 73 middot middot middot 81
︸ ︷︷ ︸
n
m V =
1 0 middot middot middot 11 0 middot middot middot 0
0 1 middot middot middot 1
︸ ︷︷ ︸
n
m
Staircase function is a stochastic function f 0 1` rarr R
0 δ 2δ (mminus1)δ mδ
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
δ = 02 σ = 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredInOrder__ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
trials=100 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
35 25 37 4631 5 32 2021 33 50 4240 27 30 312 14 48 2
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
B
Watch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
0 0 1 01 1 0 01 1 0 11 1 0 10 0 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=randommpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Prediction amp Validation
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
crosstype=2Mut=003Animmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
ctype=2StaircaseClampingAnimmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60000
100
200
300
400
500
600
700
800
900
Generation
Un
mu
tate
d lo
ci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
100
200
300
400
500
600
700
800
900
1000
Generation
Unm
uta
ted loci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Conclusion
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 38
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
How a Hyperclimbing Heuristic Works
Limit future sampling to some schemam equivalent to m
Fix defining loci of the schema in the population
Hyperclimbing Heuristic now recurses
Search occurs over the unfixed loci
And so on
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption is Weak
Staggered coarse conditional effects assumption is weakWeaker than fitness distribution assumption in the BBH
BBH assumes unstaggered coarse unconditional effects
Weaker assumptions are more likely to be satisfied in practice
Good bc we aspire to explain optimization in all GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption is Weak
Staggered coarse conditional effects assumption is weakWeaker than fitness distribution assumption in the BBH
BBH assumes unstaggered coarse unconditional effects
Weaker assumptions are more likely to be satisfied in practice
Good bc we aspire to explain optimization in all GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Proof Of Concept
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
fitness(x) simN (+025 1) if xi1 oplus xi2 oplus xi3 oplus xi4 = 1N (minus025 1) otherwise
0 Fitness
Pro
babi
lity
minus025 +025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
darr darr darr darr10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
uarr uarr uarr uarrEffective Loci
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Naive approach
Visit loci in groups of four
Check for differentiation inthe multivariate marginalfitness values
Time complexity is Ω(`4)
xi1 xi2 xi3 xi4
ExpectedMarginalFitness
0 0 0 0 minus0250 0 0 1 +0250 0 1 0 +0250 0 1 1 minus0250 1 0 0 +0250 1 0 1 minus0250 1 1 0 minus0250 1 1 1 +0251 0 0 0 +0251 0 0 1 minus0251 0 1 0 minus0251 0 1 1 +0251 1 0 0 minus0251 1 0 1 +0251 1 1 0 +0251 1 1 1 minus025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Visiting loci in groups ofthree or less wonrsquot work
xj
ExpectedMarginalFitness
0 001 00
xj1 xj2
ExpectedMarginalFitness
0 0 000 1 001 0 001 1 00
xj1 xj2 xj3
ExpectedMarginalFitness
0 0 0 000 0 1 000 1 0 000 1 1 001 0 0 001 0 1 001 1 0 001 1 1 00
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=200
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity200AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Symmetry Analytic Conclusion
Expected generations for the red dots to diverge constant wrt
Location of the red dots
Number of blue dots
Dynamics of a blue dot invariant to
Location of the red dots
Number of blue dots
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=1000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity1000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=10000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity10000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
Staircase function descriptor (m n δ σ ` LV )
L =
3 87 middot middot middot 93
42 58 middot middot middot 72
67 73 middot middot middot 81
︸ ︷︷ ︸
n
m V =
1 0 middot middot middot 11 0 middot middot middot 0
0 1 middot middot middot 1
︸ ︷︷ ︸
n
m
Staircase function is a stochastic function f 0 1` rarr R
0 δ 2δ (mminus1)δ mδ
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
δ = 02 σ = 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredInOrder__ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
trials=100 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
35 25 37 4631 5 32 2021 33 50 4240 27 30 312 14 48 2
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
B
Watch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
0 0 1 01 1 0 01 1 0 11 1 0 10 0 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=randommpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Prediction amp Validation
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
crosstype=2Mut=003Animmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
ctype=2StaircaseClampingAnimmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60000
100
200
300
400
500
600
700
800
900
Generation
Un
mu
tate
d lo
ci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
100
200
300
400
500
600
700
800
900
1000
Generation
Unm
uta
ted loci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Conclusion
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 39
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption is Weak
Staggered coarse conditional effects assumption is weakWeaker than fitness distribution assumption in the BBH
BBH assumes unstaggered coarse unconditional effects
Weaker assumptions are more likely to be satisfied in practice
Good bc we aspire to explain optimization in all GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption is Weak
Staggered coarse conditional effects assumption is weakWeaker than fitness distribution assumption in the BBH
BBH assumes unstaggered coarse unconditional effects
Weaker assumptions are more likely to be satisfied in practice
Good bc we aspire to explain optimization in all GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Proof Of Concept
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
fitness(x) simN (+025 1) if xi1 oplus xi2 oplus xi3 oplus xi4 = 1N (minus025 1) otherwise
0 Fitness
Pro
babi
lity
minus025 +025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
darr darr darr darr10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
uarr uarr uarr uarrEffective Loci
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Naive approach
Visit loci in groups of four
Check for differentiation inthe multivariate marginalfitness values
Time complexity is Ω(`4)
xi1 xi2 xi3 xi4
ExpectedMarginalFitness
0 0 0 0 minus0250 0 0 1 +0250 0 1 0 +0250 0 1 1 minus0250 1 0 0 +0250 1 0 1 minus0250 1 1 0 minus0250 1 1 1 +0251 0 0 0 +0251 0 0 1 minus0251 0 1 0 minus0251 0 1 1 +0251 1 0 0 minus0251 1 0 1 +0251 1 1 0 +0251 1 1 1 minus025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Visiting loci in groups ofthree or less wonrsquot work
xj
ExpectedMarginalFitness
0 001 00
xj1 xj2
ExpectedMarginalFitness
0 0 000 1 001 0 001 1 00
xj1 xj2 xj3
ExpectedMarginalFitness
0 0 0 000 0 1 000 1 0 000 1 1 001 0 0 001 0 1 001 1 0 001 1 1 00
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=200
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity200AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Symmetry Analytic Conclusion
Expected generations for the red dots to diverge constant wrt
Location of the red dots
Number of blue dots
Dynamics of a blue dot invariant to
Location of the red dots
Number of blue dots
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=1000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity1000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=10000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity10000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
Staircase function descriptor (m n δ σ ` LV )
L =
3 87 middot middot middot 93
42 58 middot middot middot 72
67 73 middot middot middot 81
︸ ︷︷ ︸
n
m V =
1 0 middot middot middot 11 0 middot middot middot 0
0 1 middot middot middot 1
︸ ︷︷ ︸
n
m
Staircase function is a stochastic function f 0 1` rarr R
0 δ 2δ (mminus1)δ mδ
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
δ = 02 σ = 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredInOrder__ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
trials=100 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
35 25 37 4631 5 32 2021 33 50 4240 27 30 312 14 48 2
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
B
Watch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
0 0 1 01 1 0 01 1 0 11 1 0 10 0 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=randommpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Prediction amp Validation
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
crosstype=2Mut=003Animmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
ctype=2StaircaseClampingAnimmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60000
100
200
300
400
500
600
700
800
900
Generation
Un
mu
tate
d lo
ci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
100
200
300
400
500
600
700
800
900
1000
Generation
Unm
uta
ted loci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Conclusion
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 40
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption is Weak
Staggered coarse conditional effects assumption is weakWeaker than fitness distribution assumption in the BBH
BBH assumes unstaggered coarse unconditional effects
Weaker assumptions are more likely to be satisfied in practice
Good bc we aspire to explain optimization in all GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption is Weak
Staggered coarse conditional effects assumption is weakWeaker than fitness distribution assumption in the BBH
BBH assumes unstaggered coarse unconditional effects
Weaker assumptions are more likely to be satisfied in practice
Good bc we aspire to explain optimization in all GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Proof Of Concept
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
fitness(x) simN (+025 1) if xi1 oplus xi2 oplus xi3 oplus xi4 = 1N (minus025 1) otherwise
0 Fitness
Pro
babi
lity
minus025 +025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
darr darr darr darr10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
uarr uarr uarr uarrEffective Loci
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Naive approach
Visit loci in groups of four
Check for differentiation inthe multivariate marginalfitness values
Time complexity is Ω(`4)
xi1 xi2 xi3 xi4
ExpectedMarginalFitness
0 0 0 0 minus0250 0 0 1 +0250 0 1 0 +0250 0 1 1 minus0250 1 0 0 +0250 1 0 1 minus0250 1 1 0 minus0250 1 1 1 +0251 0 0 0 +0251 0 0 1 minus0251 0 1 0 minus0251 0 1 1 +0251 1 0 0 minus0251 1 0 1 +0251 1 1 0 +0251 1 1 1 minus025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Visiting loci in groups ofthree or less wonrsquot work
xj
ExpectedMarginalFitness
0 001 00
xj1 xj2
ExpectedMarginalFitness
0 0 000 1 001 0 001 1 00
xj1 xj2 xj3
ExpectedMarginalFitness
0 0 0 000 0 1 000 1 0 000 1 1 001 0 0 001 0 1 001 1 0 001 1 1 00
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=200
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity200AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Symmetry Analytic Conclusion
Expected generations for the red dots to diverge constant wrt
Location of the red dots
Number of blue dots
Dynamics of a blue dot invariant to
Location of the red dots
Number of blue dots
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=1000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity1000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=10000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity10000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
Staircase function descriptor (m n δ σ ` LV )
L =
3 87 middot middot middot 93
42 58 middot middot middot 72
67 73 middot middot middot 81
︸ ︷︷ ︸
n
m V =
1 0 middot middot middot 11 0 middot middot middot 0
0 1 middot middot middot 1
︸ ︷︷ ︸
n
m
Staircase function is a stochastic function f 0 1` rarr R
0 δ 2δ (mminus1)δ mδ
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
δ = 02 σ = 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredInOrder__ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
trials=100 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
35 25 37 4631 5 32 2021 33 50 4240 27 30 312 14 48 2
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
B
Watch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
0 0 1 01 1 0 01 1 0 11 1 0 10 0 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=randommpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Prediction amp Validation
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
crosstype=2Mut=003Animmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
ctype=2StaircaseClampingAnimmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60000
100
200
300
400
500
600
700
800
900
Generation
Un
mu
tate
d lo
ci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
100
200
300
400
500
600
700
800
900
1000
Generation
Unm
uta
ted loci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Conclusion
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 41
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption
exist a small set of unfixed loci I such thatConditional effect of JIK is detectable
Conditional upon loci that have already been fixed
Unconditional effect of JIK may be undetectable
Eg
Effect of 10 is detectableEffect of undetectable(lsquorsquo = wildcard lsquorsquo= defined locus)
Called staggered coarse conditional effects
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption is Weak
Staggered coarse conditional effects assumption is weakWeaker than fitness distribution assumption in the BBH
BBH assumes unstaggered coarse unconditional effects
Weaker assumptions are more likely to be satisfied in practice
Good bc we aspire to explain optimization in all GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption is Weak
Staggered coarse conditional effects assumption is weakWeaker than fitness distribution assumption in the BBH
BBH assumes unstaggered coarse unconditional effects
Weaker assumptions are more likely to be satisfied in practice
Good bc we aspire to explain optimization in all GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Proof Of Concept
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
fitness(x) simN (+025 1) if xi1 oplus xi2 oplus xi3 oplus xi4 = 1N (minus025 1) otherwise
0 Fitness
Pro
babi
lity
minus025 +025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
darr darr darr darr10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
uarr uarr uarr uarrEffective Loci
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Naive approach
Visit loci in groups of four
Check for differentiation inthe multivariate marginalfitness values
Time complexity is Ω(`4)
xi1 xi2 xi3 xi4
ExpectedMarginalFitness
0 0 0 0 minus0250 0 0 1 +0250 0 1 0 +0250 0 1 1 minus0250 1 0 0 +0250 1 0 1 minus0250 1 1 0 minus0250 1 1 1 +0251 0 0 0 +0251 0 0 1 minus0251 0 1 0 minus0251 0 1 1 +0251 1 0 0 minus0251 1 0 1 +0251 1 1 0 +0251 1 1 1 minus025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Visiting loci in groups ofthree or less wonrsquot work
xj
ExpectedMarginalFitness
0 001 00
xj1 xj2
ExpectedMarginalFitness
0 0 000 1 001 0 001 1 00
xj1 xj2 xj3
ExpectedMarginalFitness
0 0 0 000 0 1 000 1 0 000 1 1 001 0 0 001 0 1 001 1 0 001 1 1 00
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=200
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity200AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Symmetry Analytic Conclusion
Expected generations for the red dots to diverge constant wrt
Location of the red dots
Number of blue dots
Dynamics of a blue dot invariant to
Location of the red dots
Number of blue dots
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=1000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity1000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=10000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity10000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
Staircase function descriptor (m n δ σ ` LV )
L =
3 87 middot middot middot 93
42 58 middot middot middot 72
67 73 middot middot middot 81
︸ ︷︷ ︸
n
m V =
1 0 middot middot middot 11 0 middot middot middot 0
0 1 middot middot middot 1
︸ ︷︷ ︸
n
m
Staircase function is a stochastic function f 0 1` rarr R
0 δ 2δ (mminus1)δ mδ
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
δ = 02 σ = 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredInOrder__ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
trials=100 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
35 25 37 4631 5 32 2021 33 50 4240 27 30 312 14 48 2
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
B
Watch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
0 0 1 01 1 0 01 1 0 11 1 0 10 0 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=randommpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Prediction amp Validation
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
crosstype=2Mut=003Animmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
ctype=2StaircaseClampingAnimmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60000
100
200
300
400
500
600
700
800
900
Generation
Un
mu
tate
d lo
ci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
100
200
300
400
500
600
700
800
900
1000
Generation
Unm
uta
ted loci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Conclusion
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 42
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption is Weak
Staggered coarse conditional effects assumption is weakWeaker than fitness distribution assumption in the BBH
BBH assumes unstaggered coarse unconditional effects
Weaker assumptions are more likely to be satisfied in practice
Good bc we aspire to explain optimization in all GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption is Weak
Staggered coarse conditional effects assumption is weakWeaker than fitness distribution assumption in the BBH
BBH assumes unstaggered coarse unconditional effects
Weaker assumptions are more likely to be satisfied in practice
Good bc we aspire to explain optimization in all GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Proof Of Concept
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
fitness(x) simN (+025 1) if xi1 oplus xi2 oplus xi3 oplus xi4 = 1N (minus025 1) otherwise
0 Fitness
Pro
babi
lity
minus025 +025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
darr darr darr darr10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
uarr uarr uarr uarrEffective Loci
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Naive approach
Visit loci in groups of four
Check for differentiation inthe multivariate marginalfitness values
Time complexity is Ω(`4)
xi1 xi2 xi3 xi4
ExpectedMarginalFitness
0 0 0 0 minus0250 0 0 1 +0250 0 1 0 +0250 0 1 1 minus0250 1 0 0 +0250 1 0 1 minus0250 1 1 0 minus0250 1 1 1 +0251 0 0 0 +0251 0 0 1 minus0251 0 1 0 minus0251 0 1 1 +0251 1 0 0 minus0251 1 0 1 +0251 1 1 0 +0251 1 1 1 minus025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Visiting loci in groups ofthree or less wonrsquot work
xj
ExpectedMarginalFitness
0 001 00
xj1 xj2
ExpectedMarginalFitness
0 0 000 1 001 0 001 1 00
xj1 xj2 xj3
ExpectedMarginalFitness
0 0 0 000 0 1 000 1 0 000 1 1 001 0 0 001 0 1 001 1 0 001 1 1 00
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=200
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity200AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Symmetry Analytic Conclusion
Expected generations for the red dots to diverge constant wrt
Location of the red dots
Number of blue dots
Dynamics of a blue dot invariant to
Location of the red dots
Number of blue dots
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=1000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity1000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=10000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity10000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
Staircase function descriptor (m n δ σ ` LV )
L =
3 87 middot middot middot 93
42 58 middot middot middot 72
67 73 middot middot middot 81
︸ ︷︷ ︸
n
m V =
1 0 middot middot middot 11 0 middot middot middot 0
0 1 middot middot middot 1
︸ ︷︷ ︸
n
m
Staircase function is a stochastic function f 0 1` rarr R
0 δ 2δ (mminus1)δ mδ
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
δ = 02 σ = 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredInOrder__ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
trials=100 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
35 25 37 4631 5 32 2021 33 50 4240 27 30 312 14 48 2
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
B
Watch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
0 0 1 01 1 0 01 1 0 11 1 0 10 0 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=randommpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Prediction amp Validation
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
crosstype=2Mut=003Animmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
ctype=2StaircaseClampingAnimmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60000
100
200
300
400
500
600
700
800
900
Generation
Un
mu
tate
d lo
ci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
100
200
300
400
500
600
700
800
900
1000
Generation
Unm
uta
ted loci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Conclusion
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 43
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Fitness Distribution Assumption is Weak
Staggered coarse conditional effects assumption is weakWeaker than fitness distribution assumption in the BBH
BBH assumes unstaggered coarse unconditional effects
Weaker assumptions are more likely to be satisfied in practice
Good bc we aspire to explain optimization in all GAs with UX
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Proof Of Concept
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
fitness(x) simN (+025 1) if xi1 oplus xi2 oplus xi3 oplus xi4 = 1N (minus025 1) otherwise
0 Fitness
Pro
babi
lity
minus025 +025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
darr darr darr darr10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
uarr uarr uarr uarrEffective Loci
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Naive approach
Visit loci in groups of four
Check for differentiation inthe multivariate marginalfitness values
Time complexity is Ω(`4)
xi1 xi2 xi3 xi4
ExpectedMarginalFitness
0 0 0 0 minus0250 0 0 1 +0250 0 1 0 +0250 0 1 1 minus0250 1 0 0 +0250 1 0 1 minus0250 1 1 0 minus0250 1 1 1 +0251 0 0 0 +0251 0 0 1 minus0251 0 1 0 minus0251 0 1 1 +0251 1 0 0 minus0251 1 0 1 +0251 1 1 0 +0251 1 1 1 minus025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Visiting loci in groups ofthree or less wonrsquot work
xj
ExpectedMarginalFitness
0 001 00
xj1 xj2
ExpectedMarginalFitness
0 0 000 1 001 0 001 1 00
xj1 xj2 xj3
ExpectedMarginalFitness
0 0 0 000 0 1 000 1 0 000 1 1 001 0 0 001 0 1 001 1 0 001 1 1 00
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=200
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity200AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Symmetry Analytic Conclusion
Expected generations for the red dots to diverge constant wrt
Location of the red dots
Number of blue dots
Dynamics of a blue dot invariant to
Location of the red dots
Number of blue dots
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=1000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity1000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=10000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity10000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
Staircase function descriptor (m n δ σ ` LV )
L =
3 87 middot middot middot 93
42 58 middot middot middot 72
67 73 middot middot middot 81
︸ ︷︷ ︸
n
m V =
1 0 middot middot middot 11 0 middot middot middot 0
0 1 middot middot middot 1
︸ ︷︷ ︸
n
m
Staircase function is a stochastic function f 0 1` rarr R
0 δ 2δ (mminus1)δ mδ
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
δ = 02 σ = 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredInOrder__ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
trials=100 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
35 25 37 4631 5 32 2021 33 50 4240 27 30 312 14 48 2
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
B
Watch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
0 0 1 01 1 0 01 1 0 11 1 0 10 0 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=randommpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Prediction amp Validation
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
crosstype=2Mut=003Animmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
ctype=2StaircaseClampingAnimmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60000
100
200
300
400
500
600
700
800
900
Generation
Un
mu
tate
d lo
ci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
100
200
300
400
500
600
700
800
900
1000
Generation
Unm
uta
ted loci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Conclusion
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 44
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Proof Of Concept
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
fitness(x) simN (+025 1) if xi1 oplus xi2 oplus xi3 oplus xi4 = 1N (minus025 1) otherwise
0 Fitness
Pro
babi
lity
minus025 +025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
darr darr darr darr10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
uarr uarr uarr uarrEffective Loci
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Naive approach
Visit loci in groups of four
Check for differentiation inthe multivariate marginalfitness values
Time complexity is Ω(`4)
xi1 xi2 xi3 xi4
ExpectedMarginalFitness
0 0 0 0 minus0250 0 0 1 +0250 0 1 0 +0250 0 1 1 minus0250 1 0 0 +0250 1 0 1 minus0250 1 1 0 minus0250 1 1 1 +0251 0 0 0 +0251 0 0 1 minus0251 0 1 0 minus0251 0 1 1 +0251 1 0 0 minus0251 1 0 1 +0251 1 1 0 +0251 1 1 1 minus025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Visiting loci in groups ofthree or less wonrsquot work
xj
ExpectedMarginalFitness
0 001 00
xj1 xj2
ExpectedMarginalFitness
0 0 000 1 001 0 001 1 00
xj1 xj2 xj3
ExpectedMarginalFitness
0 0 0 000 0 1 000 1 0 000 1 1 001 0 0 001 0 1 001 1 0 001 1 1 00
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=200
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity200AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Symmetry Analytic Conclusion
Expected generations for the red dots to diverge constant wrt
Location of the red dots
Number of blue dots
Dynamics of a blue dot invariant to
Location of the red dots
Number of blue dots
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=1000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity1000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=10000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity10000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
Staircase function descriptor (m n δ σ ` LV )
L =
3 87 middot middot middot 93
42 58 middot middot middot 72
67 73 middot middot middot 81
︸ ︷︷ ︸
n
m V =
1 0 middot middot middot 11 0 middot middot middot 0
0 1 middot middot middot 1
︸ ︷︷ ︸
n
m
Staircase function is a stochastic function f 0 1` rarr R
0 δ 2δ (mminus1)δ mδ
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
δ = 02 σ = 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredInOrder__ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
trials=100 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
35 25 37 4631 5 32 2021 33 50 4240 27 30 312 14 48 2
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
B
Watch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
0 0 1 01 1 0 01 1 0 11 1 0 10 0 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=randommpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Prediction amp Validation
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
crosstype=2Mut=003Animmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
ctype=2StaircaseClampingAnimmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60000
100
200
300
400
500
600
700
800
900
Generation
Un
mu
tate
d lo
ci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
100
200
300
400
500
600
700
800
900
1000
Generation
Unm
uta
ted loci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Conclusion
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 45
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Proof Of Concept
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
fitness(x) simN (+025 1) if xi1 oplus xi2 oplus xi3 oplus xi4 = 1N (minus025 1) otherwise
0 Fitness
Pro
babi
lity
minus025 +025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
darr darr darr darr10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
uarr uarr uarr uarrEffective Loci
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Naive approach
Visit loci in groups of four
Check for differentiation inthe multivariate marginalfitness values
Time complexity is Ω(`4)
xi1 xi2 xi3 xi4
ExpectedMarginalFitness
0 0 0 0 minus0250 0 0 1 +0250 0 1 0 +0250 0 1 1 minus0250 1 0 0 +0250 1 0 1 minus0250 1 1 0 minus0250 1 1 1 +0251 0 0 0 +0251 0 0 1 minus0251 0 1 0 minus0251 0 1 1 +0251 1 0 0 minus0251 1 0 1 +0251 1 1 0 +0251 1 1 1 minus025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Visiting loci in groups ofthree or less wonrsquot work
xj
ExpectedMarginalFitness
0 001 00
xj1 xj2
ExpectedMarginalFitness
0 0 000 1 001 0 001 1 00
xj1 xj2 xj3
ExpectedMarginalFitness
0 0 0 000 0 1 000 1 0 000 1 1 001 0 0 001 0 1 001 1 0 001 1 1 00
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=200
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity200AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Symmetry Analytic Conclusion
Expected generations for the red dots to diverge constant wrt
Location of the red dots
Number of blue dots
Dynamics of a blue dot invariant to
Location of the red dots
Number of blue dots
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=1000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity1000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=10000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity10000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
Staircase function descriptor (m n δ σ ` LV )
L =
3 87 middot middot middot 93
42 58 middot middot middot 72
67 73 middot middot middot 81
︸ ︷︷ ︸
n
m V =
1 0 middot middot middot 11 0 middot middot middot 0
0 1 middot middot middot 1
︸ ︷︷ ︸
n
m
Staircase function is a stochastic function f 0 1` rarr R
0 δ 2δ (mminus1)δ mδ
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
δ = 02 σ = 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredInOrder__ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
trials=100 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
35 25 37 4631 5 32 2021 33 50 4240 27 30 312 14 48 2
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
B
Watch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
0 0 1 01 1 0 01 1 0 11 1 0 10 0 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=randommpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Prediction amp Validation
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
crosstype=2Mut=003Animmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
ctype=2StaircaseClampingAnimmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60000
100
200
300
400
500
600
700
800
900
Generation
Un
mu
tate
d lo
ci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
100
200
300
400
500
600
700
800
900
1000
Generation
Unm
uta
ted loci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Conclusion
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 46
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Hyperclimbing Heuristic is in Good Company
Hyperclimbing an example of a global decimation heuristic
Global decimation heuristics iteratively reduce the size of asearch space
Use non-local information to find and fix partial solutionsNo backtracking
Eg Survey propagation (Merzad et al 2002)
State-of-the-art solver for large random SAT problems
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Proof Of Concept
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
fitness(x) simN (+025 1) if xi1 oplus xi2 oplus xi3 oplus xi4 = 1N (minus025 1) otherwise
0 Fitness
Pro
babi
lity
minus025 +025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
darr darr darr darr10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
uarr uarr uarr uarrEffective Loci
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Naive approach
Visit loci in groups of four
Check for differentiation inthe multivariate marginalfitness values
Time complexity is Ω(`4)
xi1 xi2 xi3 xi4
ExpectedMarginalFitness
0 0 0 0 minus0250 0 0 1 +0250 0 1 0 +0250 0 1 1 minus0250 1 0 0 +0250 1 0 1 minus0250 1 1 0 minus0250 1 1 1 +0251 0 0 0 +0251 0 0 1 minus0251 0 1 0 minus0251 0 1 1 +0251 1 0 0 minus0251 1 0 1 +0251 1 1 0 +0251 1 1 1 minus025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Visiting loci in groups ofthree or less wonrsquot work
xj
ExpectedMarginalFitness
0 001 00
xj1 xj2
ExpectedMarginalFitness
0 0 000 1 001 0 001 1 00
xj1 xj2 xj3
ExpectedMarginalFitness
0 0 0 000 0 1 000 1 0 000 1 1 001 0 0 001 0 1 001 1 0 001 1 1 00
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=200
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity200AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Symmetry Analytic Conclusion
Expected generations for the red dots to diverge constant wrt
Location of the red dots
Number of blue dots
Dynamics of a blue dot invariant to
Location of the red dots
Number of blue dots
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=1000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity1000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=10000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity10000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
Staircase function descriptor (m n δ σ ` LV )
L =
3 87 middot middot middot 93
42 58 middot middot middot 72
67 73 middot middot middot 81
︸ ︷︷ ︸
n
m V =
1 0 middot middot middot 11 0 middot middot middot 0
0 1 middot middot middot 1
︸ ︷︷ ︸
n
m
Staircase function is a stochastic function f 0 1` rarr R
0 δ 2δ (mminus1)δ mδ
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
δ = 02 σ = 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredInOrder__ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
trials=100 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
35 25 37 4631 5 32 2021 33 50 4240 27 30 312 14 48 2
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
B
Watch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
0 0 1 01 1 0 01 1 0 11 1 0 10 0 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=randommpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Prediction amp Validation
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
crosstype=2Mut=003Animmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
ctype=2StaircaseClampingAnimmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60000
100
200
300
400
500
600
700
800
900
Generation
Un
mu
tate
d lo
ci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
100
200
300
400
500
600
700
800
900
1000
Generation
Unm
uta
ted loci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Conclusion
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 47
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Proof Of Concept
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
fitness(x) simN (+025 1) if xi1 oplus xi2 oplus xi3 oplus xi4 = 1N (minus025 1) otherwise
0 Fitness
Pro
babi
lity
minus025 +025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
darr darr darr darr10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
uarr uarr uarr uarrEffective Loci
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Naive approach
Visit loci in groups of four
Check for differentiation inthe multivariate marginalfitness values
Time complexity is Ω(`4)
xi1 xi2 xi3 xi4
ExpectedMarginalFitness
0 0 0 0 minus0250 0 0 1 +0250 0 1 0 +0250 0 1 1 minus0250 1 0 0 +0250 1 0 1 minus0250 1 1 0 minus0250 1 1 1 +0251 0 0 0 +0251 0 0 1 minus0251 0 1 0 minus0251 0 1 1 +0251 1 0 0 minus0251 1 0 1 +0251 1 1 0 +0251 1 1 1 minus025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Visiting loci in groups ofthree or less wonrsquot work
xj
ExpectedMarginalFitness
0 001 00
xj1 xj2
ExpectedMarginalFitness
0 0 000 1 001 0 001 1 00
xj1 xj2 xj3
ExpectedMarginalFitness
0 0 0 000 0 1 000 1 0 000 1 1 001 0 0 001 0 1 001 1 0 001 1 1 00
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=200
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity200AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Symmetry Analytic Conclusion
Expected generations for the red dots to diverge constant wrt
Location of the red dots
Number of blue dots
Dynamics of a blue dot invariant to
Location of the red dots
Number of blue dots
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=1000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity1000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=10000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity10000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
Staircase function descriptor (m n δ σ ` LV )
L =
3 87 middot middot middot 93
42 58 middot middot middot 72
67 73 middot middot middot 81
︸ ︷︷ ︸
n
m V =
1 0 middot middot middot 11 0 middot middot middot 0
0 1 middot middot middot 1
︸ ︷︷ ︸
n
m
Staircase function is a stochastic function f 0 1` rarr R
0 δ 2δ (mminus1)δ mδ
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
δ = 02 σ = 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredInOrder__ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
trials=100 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
35 25 37 4631 5 32 2021 33 50 4240 27 30 312 14 48 2
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
B
Watch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
0 0 1 01 1 0 01 1 0 11 1 0 10 0 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=randommpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Prediction amp Validation
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
crosstype=2Mut=003Animmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
ctype=2StaircaseClampingAnimmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60000
100
200
300
400
500
600
700
800
900
Generation
Un
mu
tate
d lo
ci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
100
200
300
400
500
600
700
800
900
1000
Generation
Unm
uta
ted loci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Conclusion
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 48
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
fitness(x) simN (+025 1) if xi1 oplus xi2 oplus xi3 oplus xi4 = 1N (minus025 1) otherwise
0 Fitness
Pro
babi
lity
minus025 +025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
darr darr darr darr10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
uarr uarr uarr uarrEffective Loci
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Naive approach
Visit loci in groups of four
Check for differentiation inthe multivariate marginalfitness values
Time complexity is Ω(`4)
xi1 xi2 xi3 xi4
ExpectedMarginalFitness
0 0 0 0 minus0250 0 0 1 +0250 0 1 0 +0250 0 1 1 minus0250 1 0 0 +0250 1 0 1 minus0250 1 1 0 minus0250 1 1 1 +0251 0 0 0 +0251 0 0 1 minus0251 0 1 0 minus0251 0 1 1 +0251 1 0 0 minus0251 1 0 1 +0251 1 1 0 +0251 1 1 1 minus025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Visiting loci in groups ofthree or less wonrsquot work
xj
ExpectedMarginalFitness
0 001 00
xj1 xj2
ExpectedMarginalFitness
0 0 000 1 001 0 001 1 00
xj1 xj2 xj3
ExpectedMarginalFitness
0 0 0 000 0 1 000 1 0 000 1 1 001 0 0 001 0 1 001 1 0 001 1 1 00
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=200
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity200AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Symmetry Analytic Conclusion
Expected generations for the red dots to diverge constant wrt
Location of the red dots
Number of blue dots
Dynamics of a blue dot invariant to
Location of the red dots
Number of blue dots
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=1000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity1000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=10000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity10000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
Staircase function descriptor (m n δ σ ` LV )
L =
3 87 middot middot middot 93
42 58 middot middot middot 72
67 73 middot middot middot 81
︸ ︷︷ ︸
n
m V =
1 0 middot middot middot 11 0 middot middot middot 0
0 1 middot middot middot 1
︸ ︷︷ ︸
n
m
Staircase function is a stochastic function f 0 1` rarr R
0 δ 2δ (mminus1)δ mδ
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
δ = 02 σ = 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredInOrder__ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
trials=100 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
35 25 37 4631 5 32 2021 33 50 4240 27 30 312 14 48 2
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
B
Watch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
0 0 1 01 1 0 01 1 0 11 1 0 10 0 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=randommpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Prediction amp Validation
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
crosstype=2Mut=003Animmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
ctype=2StaircaseClampingAnimmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60000
100
200
300
400
500
600
700
800
900
Generation
Un
mu
tate
d lo
ci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
100
200
300
400
500
600
700
800
900
1000
Generation
Unm
uta
ted loci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Conclusion
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 49
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
darr darr darr darr10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
uarr uarr uarr uarrEffective Loci
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Naive approach
Visit loci in groups of four
Check for differentiation inthe multivariate marginalfitness values
Time complexity is Ω(`4)
xi1 xi2 xi3 xi4
ExpectedMarginalFitness
0 0 0 0 minus0250 0 0 1 +0250 0 1 0 +0250 0 1 1 minus0250 1 0 0 +0250 1 0 1 minus0250 1 1 0 minus0250 1 1 1 +0251 0 0 0 +0251 0 0 1 minus0251 0 1 0 minus0251 0 1 1 +0251 1 0 0 minus0251 1 0 1 +0251 1 1 0 +0251 1 1 1 minus025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Visiting loci in groups ofthree or less wonrsquot work
xj
ExpectedMarginalFitness
0 001 00
xj1 xj2
ExpectedMarginalFitness
0 0 000 1 001 0 001 1 00
xj1 xj2 xj3
ExpectedMarginalFitness
0 0 0 000 0 1 000 1 0 000 1 1 001 0 0 001 0 1 001 1 0 001 1 1 00
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=200
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity200AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Symmetry Analytic Conclusion
Expected generations for the red dots to diverge constant wrt
Location of the red dots
Number of blue dots
Dynamics of a blue dot invariant to
Location of the red dots
Number of blue dots
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=1000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity1000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=10000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity10000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
Staircase function descriptor (m n δ σ ` LV )
L =
3 87 middot middot middot 93
42 58 middot middot middot 72
67 73 middot middot middot 81
︸ ︷︷ ︸
n
m V =
1 0 middot middot middot 11 0 middot middot middot 0
0 1 middot middot middot 1
︸ ︷︷ ︸
n
m
Staircase function is a stochastic function f 0 1` rarr R
0 δ 2δ (mminus1)δ mδ
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
δ = 02 σ = 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredInOrder__ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
trials=100 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
35 25 37 4631 5 32 2021 33 50 4240 27 30 312 14 48 2
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
B
Watch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
0 0 1 01 1 0 01 1 0 11 1 0 10 0 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=randommpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Prediction amp Validation
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
crosstype=2Mut=003Animmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
ctype=2StaircaseClampingAnimmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60000
100
200
300
400
500
600
700
800
900
Generation
Un
mu
tate
d lo
ci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
100
200
300
400
500
600
700
800
900
1000
Generation
Unm
uta
ted loci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Conclusion
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 50
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
4-Bit Stochastic Learning Parities Problem
The Game
darr darr darr darr10111001011101001111000001 7rarr +0568910001010110110100011110000 7rarr minus02556511101100100010111101101110 7rarr minus03774701110101001111100000110001 7rarr minus02958900000010001000100110011101 7rarr minus1475100001100000100010001100000 7rarr minus023401000001111000111100110100 7rarr +01184411010001000101000011000110 7rarr +03148100000011100010001100000111 7rarr +1443500100111000111000001110000 7rarr minus03509710100011011010101010100001 7rarr +06232300011011101010100010100000 7rarr +07990500101010101100101110100000 7rarr +09408901010110000000110110110011 7rarr minus09920911100101000110010110110101 7rarr +02120400011111011101110101000111 7rarr +02378800001111111101011110111010 7rarr minus10078
00001011101111000000111100 7rarr +1082311011100111100010100111101 7rarr minus01315
uarr uarr uarr uarrEffective Loci
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Naive approach
Visit loci in groups of four
Check for differentiation inthe multivariate marginalfitness values
Time complexity is Ω(`4)
xi1 xi2 xi3 xi4
ExpectedMarginalFitness
0 0 0 0 minus0250 0 0 1 +0250 0 1 0 +0250 0 1 1 minus0250 1 0 0 +0250 1 0 1 minus0250 1 1 0 minus0250 1 1 1 +0251 0 0 0 +0251 0 0 1 minus0251 0 1 0 minus0251 0 1 1 +0251 1 0 0 minus0251 1 0 1 +0251 1 1 0 +0251 1 1 1 minus025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Visiting loci in groups ofthree or less wonrsquot work
xj
ExpectedMarginalFitness
0 001 00
xj1 xj2
ExpectedMarginalFitness
0 0 000 1 001 0 001 1 00
xj1 xj2 xj3
ExpectedMarginalFitness
0 0 0 000 0 1 000 1 0 000 1 1 001 0 0 001 0 1 001 1 0 001 1 1 00
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=200
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity200AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Symmetry Analytic Conclusion
Expected generations for the red dots to diverge constant wrt
Location of the red dots
Number of blue dots
Dynamics of a blue dot invariant to
Location of the red dots
Number of blue dots
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=1000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity1000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=10000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity10000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
Staircase function descriptor (m n δ σ ` LV )
L =
3 87 middot middot middot 93
42 58 middot middot middot 72
67 73 middot middot middot 81
︸ ︷︷ ︸
n
m V =
1 0 middot middot middot 11 0 middot middot middot 0
0 1 middot middot middot 1
︸ ︷︷ ︸
n
m
Staircase function is a stochastic function f 0 1` rarr R
0 δ 2δ (mminus1)δ mδ
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
δ = 02 σ = 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredInOrder__ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
trials=100 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
35 25 37 4631 5 32 2021 33 50 4240 27 30 312 14 48 2
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
B
Watch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
0 0 1 01 1 0 01 1 0 11 1 0 10 0 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=randommpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Prediction amp Validation
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
crosstype=2Mut=003Animmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
ctype=2StaircaseClampingAnimmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60000
100
200
300
400
500
600
700
800
900
Generation
Un
mu
tate
d lo
ci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
100
200
300
400
500
600
700
800
900
1000
Generation
Unm
uta
ted loci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Conclusion
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 51
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Naive approach
Visit loci in groups of four
Check for differentiation inthe multivariate marginalfitness values
Time complexity is Ω(`4)
xi1 xi2 xi3 xi4
ExpectedMarginalFitness
0 0 0 0 minus0250 0 0 1 +0250 0 1 0 +0250 0 1 1 minus0250 1 0 0 +0250 1 0 1 minus0250 1 1 0 minus0250 1 1 1 +0251 0 0 0 +0251 0 0 1 minus0251 0 1 0 minus0251 0 1 1 +0251 1 0 0 minus0251 1 0 1 +0251 1 1 0 +0251 1 1 1 minus025
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Visiting loci in groups ofthree or less wonrsquot work
xj
ExpectedMarginalFitness
0 001 00
xj1 xj2
ExpectedMarginalFitness
0 0 000 1 001 0 001 1 00
xj1 xj2 xj3
ExpectedMarginalFitness
0 0 0 000 0 1 000 1 0 000 1 1 001 0 0 001 0 1 001 1 0 001 1 1 00
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=200
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity200AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Symmetry Analytic Conclusion
Expected generations for the red dots to diverge constant wrt
Location of the red dots
Number of blue dots
Dynamics of a blue dot invariant to
Location of the red dots
Number of blue dots
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=1000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity1000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=10000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity10000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
Staircase function descriptor (m n δ σ ` LV )
L =
3 87 middot middot middot 93
42 58 middot middot middot 72
67 73 middot middot middot 81
︸ ︷︷ ︸
n
m V =
1 0 middot middot middot 11 0 middot middot middot 0
0 1 middot middot middot 1
︸ ︷︷ ︸
n
m
Staircase function is a stochastic function f 0 1` rarr R
0 δ 2δ (mminus1)δ mδ
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
δ = 02 σ = 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredInOrder__ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
trials=100 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
35 25 37 4631 5 32 2021 33 50 4240 27 30 312 14 48 2
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
B
Watch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
0 0 1 01 1 0 01 1 0 11 1 0 10 0 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=randommpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Prediction amp Validation
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
crosstype=2Mut=003Animmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
ctype=2StaircaseClampingAnimmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60000
100
200
300
400
500
600
700
800
900
Generation
Un
mu
tate
d lo
ci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
100
200
300
400
500
600
700
800
900
1000
Generation
Unm
uta
ted loci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Conclusion
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 52
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Solving the 4-Bit Stochastic Learning Parities Problem
Visiting loci in groups ofthree or less wonrsquot work
xj
ExpectedMarginalFitness
0 001 00
xj1 xj2
ExpectedMarginalFitness
0 0 000 1 001 0 001 1 00
xj1 xj2 xj3
ExpectedMarginalFitness
0 0 0 000 0 1 000 1 0 000 1 1 001 0 0 001 0 1 001 1 0 001 1 1 00
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=200
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity200AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Symmetry Analytic Conclusion
Expected generations for the red dots to diverge constant wrt
Location of the red dots
Number of blue dots
Dynamics of a blue dot invariant to
Location of the red dots
Number of blue dots
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=1000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity1000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=10000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity10000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
Staircase function descriptor (m n δ σ ` LV )
L =
3 87 middot middot middot 93
42 58 middot middot middot 72
67 73 middot middot middot 81
︸ ︷︷ ︸
n
m V =
1 0 middot middot middot 11 0 middot middot middot 0
0 1 middot middot middot 1
︸ ︷︷ ︸
n
m
Staircase function is a stochastic function f 0 1` rarr R
0 δ 2δ (mminus1)δ mδ
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
δ = 02 σ = 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredInOrder__ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
trials=100 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
35 25 37 4631 5 32 2021 33 50 4240 27 30 312 14 48 2
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
B
Watch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
0 0 1 01 1 0 01 1 0 11 1 0 10 0 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=randommpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Prediction amp Validation
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
crosstype=2Mut=003Animmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
ctype=2StaircaseClampingAnimmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60000
100
200
300
400
500
600
700
800
900
Generation
Un
mu
tate
d lo
ci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
100
200
300
400
500
600
700
800
900
1000
Generation
Unm
uta
ted loci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Conclusion
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 53
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=200
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity200AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Symmetry Analytic Conclusion
Expected generations for the red dots to diverge constant wrt
Location of the red dots
Number of blue dots
Dynamics of a blue dot invariant to
Location of the red dots
Number of blue dots
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=1000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity1000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=10000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity10000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
Staircase function descriptor (m n δ σ ` LV )
L =
3 87 middot middot middot 93
42 58 middot middot middot 72
67 73 middot middot middot 81
︸ ︷︷ ︸
n
m V =
1 0 middot middot middot 11 0 middot middot middot 0
0 1 middot middot middot 1
︸ ︷︷ ︸
n
m
Staircase function is a stochastic function f 0 1` rarr R
0 δ 2δ (mminus1)δ mδ
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
δ = 02 σ = 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredInOrder__ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
trials=100 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
35 25 37 4631 5 32 2021 33 50 4240 27 30 312 14 48 2
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
B
Watch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
0 0 1 01 1 0 01 1 0 11 1 0 10 0 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=randommpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Prediction amp Validation
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
crosstype=2Mut=003Animmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
ctype=2StaircaseClampingAnimmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60000
100
200
300
400
500
600
700
800
900
Generation
Un
mu
tate
d lo
ci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
100
200
300
400
500
600
700
800
900
1000
Generation
Unm
uta
ted loci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Conclusion
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 54
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Symmetry Analytic Conclusion
Expected generations for the red dots to diverge constant wrt
Location of the red dots
Number of blue dots
Dynamics of a blue dot invariant to
Location of the red dots
Number of blue dots
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=1000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity1000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=10000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity10000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
Staircase function descriptor (m n δ σ ` LV )
L =
3 87 middot middot middot 93
42 58 middot middot middot 72
67 73 middot middot middot 81
︸ ︷︷ ︸
n
m V =
1 0 middot middot middot 11 0 middot middot middot 0
0 1 middot middot middot 1
︸ ︷︷ ︸
n
m
Staircase function is a stochastic function f 0 1` rarr R
0 δ 2δ (mminus1)δ mδ
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
δ = 02 σ = 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredInOrder__ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
trials=100 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
35 25 37 4631 5 32 2021 33 50 4240 27 30 312 14 48 2
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
B
Watch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
0 0 1 01 1 0 01 1 0 11 1 0 10 0 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=randommpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Prediction amp Validation
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
crosstype=2Mut=003Animmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
ctype=2StaircaseClampingAnimmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60000
100
200
300
400
500
600
700
800
900
Generation
Un
mu
tate
d lo
ci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
100
200
300
400
500
600
700
800
900
1000
Generation
Unm
uta
ted loci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Conclusion
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 55
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=1000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity1000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=10000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity10000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
Staircase function descriptor (m n δ σ ` LV )
L =
3 87 middot middot middot 93
42 58 middot middot middot 72
67 73 middot middot middot 81
︸ ︷︷ ︸
n
m V =
1 0 middot middot middot 11 0 middot middot middot 0
0 1 middot middot middot 1
︸ ︷︷ ︸
n
m
Staircase function is a stochastic function f 0 1` rarr R
0 δ 2δ (mminus1)δ mδ
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
δ = 02 σ = 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredInOrder__ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
trials=100 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
35 25 37 4631 5 32 2021 33 50 4240 27 30 312 14 48 2
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
B
Watch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
0 0 1 01 1 0 01 1 0 11 1 0 10 0 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=randommpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Prediction amp Validation
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
crosstype=2Mut=003Animmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
ctype=2StaircaseClampingAnimmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60000
100
200
300
400
500
600
700
800
900
Generation
Un
mu
tate
d lo
ci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
100
200
300
400
500
600
700
800
900
1000
Generation
Unm
uta
ted loci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Conclusion
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 56
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
BWatch On YouTube
Dynamics of a SGA on the 4-Bit Learning Parities problemLoci=10000
Keki Burjorjee Explaining Optimization in GAs with UX
4BitParity10000AttrsDissmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
Staircase function descriptor (m n δ σ ` LV )
L =
3 87 middot middot middot 93
42 58 middot middot middot 72
67 73 middot middot middot 81
︸ ︷︷ ︸
n
m V =
1 0 middot middot middot 11 0 middot middot middot 0
0 1 middot middot middot 1
︸ ︷︷ ︸
n
m
Staircase function is a stochastic function f 0 1` rarr R
0 δ 2δ (mminus1)δ mδ
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
δ = 02 σ = 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredInOrder__ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
trials=100 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
35 25 37 4631 5 32 2021 33 50 4240 27 30 312 14 48 2
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
B
Watch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
0 0 1 01 1 0 01 1 0 11 1 0 10 0 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=randommpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Prediction amp Validation
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
crosstype=2Mut=003Animmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
ctype=2StaircaseClampingAnimmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60000
100
200
300
400
500
600
700
800
900
Generation
Un
mu
tate
d lo
ci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
100
200
300
400
500
600
700
800
900
1000
Generation
Unm
uta
ted loci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Conclusion
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 57
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
Staircase function descriptor (m n δ σ ` LV )
L =
3 87 middot middot middot 93
42 58 middot middot middot 72
67 73 middot middot middot 81
︸ ︷︷ ︸
n
m V =
1 0 middot middot middot 11 0 middot middot middot 0
0 1 middot middot middot 1
︸ ︷︷ ︸
n
m
Staircase function is a stochastic function f 0 1` rarr R
0 δ 2δ (mminus1)δ mδ
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
δ = 02 σ = 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredInOrder__ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
trials=100 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
35 25 37 4631 5 32 2021 33 50 4240 27 30 312 14 48 2
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
B
Watch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
0 0 1 01 1 0 01 1 0 11 1 0 10 0 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=randommpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Prediction amp Validation
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
crosstype=2Mut=003Animmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
ctype=2StaircaseClampingAnimmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60000
100
200
300
400
500
600
700
800
900
Generation
Un
mu
tate
d lo
ci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
100
200
300
400
500
600
700
800
900
1000
Generation
Unm
uta
ted loci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Conclusion
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 58
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
δ = 02 σ = 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredInOrder__ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
trials=100 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
35 25 37 4631 5 32 2021 33 50 4240 27 30 312 14 48 2
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
B
Watch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
0 0 1 01 1 0 01 1 0 11 1 0 10 0 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=randommpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Prediction amp Validation
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
crosstype=2Mut=003Animmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
ctype=2StaircaseClampingAnimmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60000
100
200
300
400
500
600
700
800
900
Generation
Un
mu
tate
d lo
ci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
100
200
300
400
500
600
700
800
900
1000
Generation
Unm
uta
ted loci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Conclusion
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 59
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
δ = 02 σ = 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredInOrder__ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
trials=100 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
35 25 37 4631 5 32 2021 33 50 4240 27 30 312 14 48 2
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
B
Watch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
0 0 1 01 1 0 01 1 0 11 1 0 10 0 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=randommpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Prediction amp Validation
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
crosstype=2Mut=003Animmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
ctype=2StaircaseClampingAnimmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60000
100
200
300
400
500
600
700
800
900
Generation
Un
mu
tate
d lo
ci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
100
200
300
400
500
600
700
800
900
1000
Generation
Unm
uta
ted loci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Conclusion
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 60
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
δ = 02 σ = 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredInOrder__ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
trials=100 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
35 25 37 4631 5 32 2021 33 50 4240 27 30 312 14 48 2
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
B
Watch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
0 0 1 01 1 0 01 1 0 11 1 0 10 0 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=randommpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Prediction amp Validation
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
crosstype=2Mut=003Animmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
ctype=2StaircaseClampingAnimmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60000
100
200
300
400
500
600
700
800
900
Generation
Un
mu
tate
d lo
ci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
100
200
300
400
500
600
700
800
900
1000
Generation
Unm
uta
ted loci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Conclusion
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 61
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Function
64 128 192 256
64
128
192
256
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
δ = 02 σ = 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredInOrder__ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
trials=100 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
35 25 37 4631 5 32 2021 33 50 4240 27 30 312 14 48 2
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
B
Watch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
0 0 1 01 1 0 01 1 0 11 1 0 10 0 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=randommpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Prediction amp Validation
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
crosstype=2Mut=003Animmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
ctype=2StaircaseClampingAnimmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60000
100
200
300
400
500
600
700
800
900
Generation
Un
mu
tate
d lo
ci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
100
200
300
400
500
600
700
800
900
1000
Generation
Unm
uta
ted loci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Conclusion
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 62
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Staircase Functions
δ = 02 σ = 1
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredInOrder__ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
trials=100 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
35 25 37 4631 5 32 2021 33 50 4240 27 30 312 14 48 2
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
B
Watch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
0 0 1 01 1 0 01 1 0 11 1 0 10 0 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=randommpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Prediction amp Validation
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
crosstype=2Mut=003Animmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
ctype=2StaircaseClampingAnimmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60000
100
200
300
400
500
600
700
800
900
Generation
Un
mu
tate
d lo
ci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
100
200
300
400
500
600
700
800
900
1000
Generation
Unm
uta
ted loci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Conclusion
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 63
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredInOrder__ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
trials=100 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
35 25 37 4631 5 32 2021 33 50 4240 27 30 312 14 48 2
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
B
Watch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
0 0 1 01 1 0 01 1 0 11 1 0 10 0 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=randommpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Prediction amp Validation
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
crosstype=2Mut=003Animmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
ctype=2StaircaseClampingAnimmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60000
100
200
300
400
500
600
700
800
900
Generation
Un
mu
tate
d lo
ci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
100
200
300
400
500
600
700
800
900
1000
Generation
Unm
uta
ted loci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Conclusion
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 64
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
1 2 3 45 6 7 89 10 11 12
13 14 15 1617 18 19 20
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
trials=100 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
35 25 37 4631 5 32 2021 33 50 4240 27 30 312 14 48 2
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
B
Watch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
0 0 1 01 1 0 01 1 0 11 1 0 10 0 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=randommpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Prediction amp Validation
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
crosstype=2Mut=003Animmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
ctype=2StaircaseClampingAnimmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60000
100
200
300
400
500
600
700
800
900
Generation
Un
mu
tate
d lo
ci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
100
200
300
400
500
600
700
800
900
1000
Generation
Unm
uta
ted loci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Conclusion
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 65
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 50
m = 5
n = 4
δ = 02
σ = 1
L =
35 25 37 4631 5 32 2021 33 50 4240 27 30 312 14 48 2
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
B
Watch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=50_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
0 0 1 01 1 0 01 1 0 11 1 0 10 0 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=randommpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Prediction amp Validation
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
crosstype=2Mut=003Animmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
ctype=2StaircaseClampingAnimmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60000
100
200
300
400
500
600
700
800
900
Generation
Un
mu
tate
d lo
ci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
100
200
300
400
500
600
700
800
900
1000
Generation
Unm
uta
ted loci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Conclusion
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 66
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
1 1 1 11 1 1 11 1 1 11 1 1 11 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=onesmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
0 0 1 01 1 0 01 1 0 11 1 0 10 0 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=randommpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Prediction amp Validation
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
crosstype=2Mut=003Animmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
ctype=2StaircaseClampingAnimmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60000
100
200
300
400
500
600
700
800
900
Generation
Un
mu
tate
d lo
ci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
100
200
300
400
500
600
700
800
900
1000
Generation
Unm
uta
ted loci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Conclusion
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 67
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 5000
m = 5
n = 4
δ = 02
σ = 1
L =2050 4659 1931 32842130 2404 205 4418143 1284 53 437
2061 904 2650 30803503 724 4822 4444
V =
0 0 1 01 1 0 01 1 0 11 1 0 10 0 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
coloredRandom___ell=5000_V=randommpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Prediction amp Validation
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
crosstype=2Mut=003Animmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
ctype=2StaircaseClampingAnimmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60000
100
200
300
400
500
600
700
800
900
Generation
Un
mu
tate
d lo
ci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
100
200
300
400
500
600
700
800
900
1000
Generation
Unm
uta
ted loci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Conclusion
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 68
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Prediction amp Validation
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
crosstype=2Mut=003Animmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
ctype=2StaircaseClampingAnimmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60000
100
200
300
400
500
600
700
800
900
Generation
Un
mu
tate
d lo
ci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
100
200
300
400
500
600
700
800
900
1000
Generation
Unm
uta
ted loci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Conclusion
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 69
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
crosstype=2Mut=003Animmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
ctype=2StaircaseClampingAnimmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60000
100
200
300
400
500
600
700
800
900
Generation
Un
mu
tate
d lo
ci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
100
200
300
400
500
600
700
800
900
1000
Generation
Unm
uta
ted loci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Conclusion
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 70
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
ctype=2StaircaseClampingAnimmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60000
100
200
300
400
500
600
700
800
900
Generation
Un
mu
tate
d lo
ci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
100
200
300
400
500
600
700
800
900
1000
Generation
Unm
uta
ted loci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Conclusion
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 71
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
ctype=2StaircaseClampingAnimmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60000
100
200
300
400
500
600
700
800
900
Generation
Un
mu
tate
d lo
ci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
100
200
300
400
500
600
700
800
900
1000
Generation
Unm
uta
ted loci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Conclusion
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 72
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
ctype=2StaircaseClampingAnimmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60000
100
200
300
400
500
600
700
800
900
Generation
Un
mu
tate
d lo
ci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
100
200
300
400
500
600
700
800
900
1000
Generation
Unm
uta
ted loci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Conclusion
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 73
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
` = 500
m = 125
n = 4
δ = 02
σ = 1
L =1 2 3 45 6 7 8
496 497 498 500
V =
1 1 1 11 1 1 11 1 1 1
1 1 1 1
BWatch On YouTube
Keki Burjorjee Explaining Optimization in GAs with UX
ctype=2StaircaseClampingAnimmpg
Media File (videompeg)
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60000
100
200
300
400
500
600
700
800
900
Generation
Un
mu
tate
d lo
ci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
100
200
300
400
500
600
700
800
900
1000
Generation
Unm
uta
ted loci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Conclusion
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 74
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60000
100
200
300
400
500
600
700
800
900
Generation
Un
mu
tate
d lo
ci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
100
200
300
400
500
600
700
800
900
1000
Generation
Unm
uta
ted loci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Conclusion
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 75
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60000
100
200
300
400
500
600
700
800
900
Generation
Un
mu
tate
d lo
ci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
100
200
300
400
500
600
700
800
900
1000
Generation
Unm
uta
ted loci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Conclusion
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 76
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Uniform Random MAX-3SAT
variables=1000 clauses=4000
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60003550
3600
3650
3700
3750
3800
3850
3900
3950
4000
Generations
Satisfied c
lauses
0 2000 4000 60000
100
200
300
400
500
600
700
800
900
Generation
Un
mu
tate
d lo
ci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
100
200
300
400
500
600
700
800
900
1000
Generation
Unm
uta
ted loci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Conclusion
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 77
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
100
200
300
400
500
600
700
800
900
1000
Generation
Unm
uta
ted loci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Conclusion
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 78
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
100
200
300
400
500
600
700
800
900
1000
Generation
Unm
uta
ted loci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Conclusion
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 79
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
SK Spin Glasses System
spins=1000drawn from minus1+1 fitness(σ) =
sum1leiltjle`
Jijσiσj Jij drawn from N (0 1)
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
05
1
15
2
25x 10
4
Generations
Fitness
0 2000 4000 6000 80000
100
200
300
400
500
600
700
800
900
1000
Generation
Unm
uta
ted loci
trials=10 Error bars show standard error
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Conclusion
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 80
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Conclusion
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 81
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 82
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 83
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 84
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 85
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Summary
Proposed the Hyperclimbing Hypothesis
Weak assumptions about the distribution of fitness
Based on a computational efficiency of GAs with UX
Upfront proof of concept
Testable predictions + Validation on
Uniform Random MAX-3SATSK Spin Glasses Problem
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 86
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 87
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Corollaries
Implicit Parallelism is real
Not your grandmotherrsquos implicit parallelism
Effect of coarse schema partitions evaluated in parallel
NOT the average fitness of short schemata
Defining length of a schema partition is immaterial
New Implicit parallelism more powerful
Think in terms of Hyperscapes not Landscapes
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 88
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 89
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 90
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 91
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 92
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
What Now
Flesh out the Hypothesis
What does a deceptive hyperscape look like
More Testable Predictions + Validation
Are staggered coarse conditional effects indeed common
Generalization
Explain optimization in other EAs
Outreach
Identify and efficiently solve problems familiar totheoretical computer scientists
Exploitation
Construct better representationsChoose better parameters for existing EAsConstruct better EAs (eg backtracking EAs)
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion Page 93
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion
Questions
follow me evohackr
Keki Burjorjee Explaining Optimization in GAs with UX
Introduction Hyperclimbing Proof of Concept Prediction amp Validation Conclusion 
