CAP6938Neuroevolution and Artificial Embryogeny

Leaky Integrator Neurons and CTRNNs

Dr. Kenneth Stanley

March 6, 2006

Artificial Neurons are a Model

• Standard activation model

• But a real neuron doesn’t have an activation level– Real neurons fire in spike trains

– Spikes/second is a rate– Therefore, standard activation can be thought of as outputting a

firing rate at discrete timesteps (i.e. rate encoding)

n

iijij wxH

1

Wolfgang Maass, http://www.tu-graz.ac.at/igi/maass

What is Lost in Rate Encoding?

• Timing information

• Synchronization

• Activity between discrete timesteps

<>                                                                                                                                        

30 Neurons Firing in a monkey’s striate cortexFrom Krüger and Aiple [Krüger and Aiple, 1988]. Reprinted from www.igi.tugraz.at/ maass/123/node2.html

Spikes Can Be Encoded Explicitly

• Leaky integrate and fire neurons• Encode each individual spike• Time is represented exactly• Each spike has an associated time• The timing of recent incoming spikes determines

whether a neuron will fire• Computationally expensive• Can we do almost as well without encoding

every single spike?

Yes: Leaky Integrator Neurons (CTRNNS; Continuous Time Recurrent Neural Networks)

• Idea: Calculate activation at discrete steps but describe rate of change on a continuous scale

• Instead of activating only based on input, include a temporal component of activation that controls the rate at which activation goes up or down

• Then the neuron can react to changes in a temporal manner, like spikes

Activation Rate Builds and Decays

• Incoming activation causes the output level to climb over time

• We can sample the rate at any discrete granularity desired

• A view is created of temporal dynamics without full spike-event simulation

ActivationLevel(i.e. spike rate)

time

Input to neuron

Output over time

What is Leaking In a Leaky Integrator?

• The neuron loses potential at a defined rate• Each neuron leaks at its own constant rate• Each neuron integrates at the same constant

rate as well

ActivationLevel(i.e. spike rate)

time

Leaking activation level (membrane potential)

Leaky Integrator Equations

• Expressing rate of change of activation level:

• Apply Euler Integration to derive discrete-time equivalent

• Expressing current activation in terms of activation on previous discrete timestep:

Leak

Real timeBetweensteps

Equations from: Blynel, J., and Floreano, D. (2002). Levels of dynamics neural controllers. In Proceedings of the Seventh International Behavior on From Animals to Animats, 272–281.

What Can a CTRNN Do?

• With the right time constants for each neuron, complex temporal patterns can be generated

• That is, the time constants are a new parameter (inside nodes) that can evolve

• More powerful than a regular RNN

• Capable of generating complex tenporal patterns with no input and no clock

Pattern Generation for What?

• Walking gaits with no input!

Evolution of central pattern generators for bipedal walking in a real-time physics environment

T Reil, P Husbands - Evolutionary Computation, IEEE Transactions on, 2002

Reil and Husbands Went on to Found the Company NaturalMotion

Pattern Generation for What?

• Salamander walking gait

• Wing flapping

Ijspeert A.J.: A connectionist central pattern generator for the aquatic and terrestrial gaits of a simulated salamander, Biological Cybernetics, Vol. 84:5, 2001, pp 331-348.

Evolution of neuro-controllers for flapping-wing animats - group of 2 »JB Mouret, S Doncieux, L Muratet, T Druot, JA … - Proceedings of the Journees MicroDrones, Toulouse, 2004

Maybe Good for Other Things with Temporal Patterning

• Music?• Picture Drawing? (certain types of patterns)• Tasks that we typically do not conceive in

terms of patterns?• Learning tasks (better than a simple RNN?;

Blynel and Floreano 2002 paper)• Largely unexplored• How far away from the benefits of a true

spiking model?

Leaky NEAT

• There is a rough, largely untested leakyNEAT at the NEAT Users Group files section:– http://groups.yahoo.com/group/neat/files/ – Introduces a new activation function and new

time constant parameter in the nodes

• The topology of most CTRNNs in the past was determined completely by the researcher

Homework due 3/8/06 (see next slide)

Next Topic: Non-neural NEAT, Closing Remarks on Survey Portion of

Class • Complexification and protection of innovation in non-neural

structures• Example: Cellular Automata neighborhood functions• What have we learned, what is its significance, and where does the

field stand?

Read for 3/8/06: Mitchell Textbook pp. 44-55 (Evolving Cellular Automata) think about: How would NEAT apply to this task?

Homework Due 3/8/06Genetic operators all working: •Mating two genomes: mate_multipoint, mate_multipoint_avg, others

•Compatibility measuring: return distance of two genomes from each other based on coefficients in compatibility equation and historical markings

•Structural mutations: mutate_add_link, mutate_add_node, others

•Weight/parameter mutations: mutate_link_weights, mutating other parameters

•Special mutations: mutate_link_enable_toggle (toggle enable flag), etc.

•Special restrictions: control probability of certain types of mutations such as adding a recurrent connection vs. a feedforward connection

Turn in summary, code, and examples demonstrating that all functions work. Must include checks that phenotypes from genotypes that are new or altered are created properly and work.

Project Milestones (25% of grade)

• 2/6: Initial proposal and project description• 2/15: Domain and phenotype code and examples• 2/27: Genes and Genotype to Phenotype mapping • 3/8: Genetic operators all working• 3/27: Population level and main loop working• 4/10: Final project and presentation due (75% of grade)