evolution of synchronization and desynchronization in digital...

Evolution of Synchronizationand Desynchronization inDigital Organisms

* Contact author.** Department of Computer Science and Engineering, Michigan State University, 3115 EnE-mail: [email protected] (D.B.K.); [email protected] (P.K.M.)

© 2011 Massachusetts Institute of Technology

David B. Knoester**Michigan State University

Philip K. McKinley*,**Michigan State University

KeywordsEvolutionary computation, synchronization,desynchronization, cooperation, digitalevolution, Avida platform

Abstract We present a study in the evolution of temporal behavior,specifically synchronization and desynchronization, through digitalevolution and group selection. In digital evolution, a populationof self-replicating computer programs exists in a user-definedcomputational environment and is subject to instruction-levelmutations and natural selection. Group selection links the survivalof the individual to the survival of its group, thus encouragingcooperation. Previous approaches to engineering synchronization anddesynchronization algorithms have taken inspiration from nature:In the well-known firefly model, the only form of communicationbetween agents is in the form of flash messages among neighbors.Here we demonstrate that populations of digital organisms, providedwith a similar mechanism and minimal information about theirenvironment, are capable of evolving algorithms for synchronizationand desynchronization, and that the evolved behaviors are robustto message loss. We further describe how the evolved behavior forsynchronization mimics that of the well-known Ermentrout modelfor firefly synchronization in biology. In addition to discoveringself-organizing behaviors for distributed computing systems, thisresult indicates that digital evolution may be used to further ourunderstanding of synchronization in biology.

1 Introduction

The natural world is replete with organisms that exhibit cooperative behaviors of varying complexity.For example, nearly all species of microorganism cooperate to form extracellular structures called bio-films [10], and many social insects cooperate to build complex nests [15]. Some cooperative behaviorsexhibit synchrony. For example, honeybees synchronize their activity cycles [35], fiddler crabs synchro-nously wave their oversized claw [3], and ants synchronize their alarm drumming [16]. Humans arealso known to exhibit synchrony, for example, neuronal activity within the cerebral cortex [32] andfor social coordination [31]. One of the more striking examples of synchrony in the natural world isthe coordinated flashing of male fireflies. In some parts of southeast Asia, these fireflies synchronizetheir flashes to a common period and phase over a distance of many miles [5].

gineering Building, East Lansing, MI 48824-1027.

Artificial Life 17: 1–20 (2011)

D. B. Knoester and P. K. McKinley Evolution of Synchronization and Desynchronization

Temporal behaviors are also important in computational settings. For example, two topics of recentinterest in distributed systems are heartbeat synchronization and desynchronization. In heartbeat syn-chronization, nodes in a distributed system are required to generate periodic heartbeat signals at approxi-mately the same time [2], while in desynchronization, nodes with the same period are to spread theirsignals in phase space, similarly to round-robin scheduling [33]. Heartbeat synchronization is a commonpart of communication protocols that require restarts [18], and desynchronization can play an importantrole in medium access control, fair scheduling, and energy efficiency [33].

Similarly to how fireflies have evolved to synchronize their flashes, here we use digital evolution [28], aform of evolutionary computation, to evolve digital organisms that both synchronize and desynchronizetheir virtual flashes in response to different selection pressures. In digital evolution, a population ofdigital organisms exists in a user-defined computational environment. These organisms replicate, com-pete for resources, and are subject to mutation and natural selection. Over thousands of generations,they can evolve to survive, and even thrive, under extremely dynamic and adverse conditions. In thisstudy we use Avida [29], a digital evolution platform previously used to study the evolution of complex-ity [1, 23] and other biological phenomena. Digital organisms in Avida exist in a spatial environment,communicate with their neighbors, and execute their genome (a list of virtual CPU instructions), with theimportant distinction that their program undergoes evolution. Recently, Avida has been applied to anincreasingly diverse set of engineering problems, including the discovery of novel algorithms for leaderelection [21], construction of communication networks [22], and consensus [19].

To evolve synchronization and desynchronization, we extended Avida with a small set of instructionsthat enable the transmission and reception of virtual flashes. For evolving cooperative behavior, wemade use of a framework within Avida that enables group selection, and we defined fitness functionsthat reward groups of digital organisms for exhibiting synchronization and desynchronization behaviors.Specifically, we evolved populations that, when initialized to a random desynchronized state, will rapidlysynchronize, and we evolved populations that, when initialized to a synchronized state, will rapidly de-synchronize. We analyzed the evolved genomes and found that the solutions for both synchronizationand desynchronization (those that performed best according to our fitness functions) utilize complex,local frequency-advance and frequency-delay strategies from which the collective behaviors emerge.

The contributions of this research are as follows. First, we demonstrate the evolution of synchro-nization and desynchronization behavior in digital organisms. Second, we investigate whether these be-haviors are resilient to failures in the transmission of virtual flashes. Third, we perform an in-depthanalysis of the genomes responsible for these behaviors and, in the case of synchronization, find thatthe evolved behavior uses a strategy remarkably similar to models of firefly synchronization. Finally,these results demonstrate that the Avida platform for digital evolution offers a promising approachto the design of behaviors for distributed computing systems. In contrast to engineered systems, evolu-tionary approaches enable the discovery of local behaviors that give rise to the desired global behavior,without bias that may be introduced by humans. Our research explores the role digital evolution can playin designing distributed systems that remain effective under dynamic conditions. While evolved solu-tions may share some of the inherent imperfections of natural organisms, they might also be resilient tounexpected conditions, in much the same way that living organisms adapt to their environment.

A preliminary investigation of synchronization was presented as a poster at the European Con-ference on Artificial Life (ECAL) in 2009 [20]. This article contains additional results and analysis ofsynchronization, and presents our study of desynchronization for the first time.

2 Related Work

Examples of cooperative behavior are pervasive in biological systems, and include cooperation amongmicroorganisms and insects, and even the schooling of fish to avoid predators. Recent bio-inspiredstudies of temporal behaviors include the work of Babaoglu [2], Patel et al. [33], and Christensenet al. [7], whose studies of synchronization and desynchronization were in part inspired by the syn-chronization of firefly flashes. Their proposed algorithms were based on the mathematical modeling

2 Artificial Life Volume 17, Number 1


of pulse-coupled oscillators by Mirollo and Strogatz [25], as well as specific models of firefly synchro-nization by Ermentrout [12]. Babaoglu and colleagues designed a protocol for heartbeat synchronizationthat was based on the Ermentrout model, and showed that this mechanism was remarkably effective atsynchronization, even in the presence of message loss. Patel and colleagues described two differentprotocols for desynchronization, one based on particle diffusion and another based on firefly synchro-nization. Christensen and colleagues designed a series of synchronization algorithms for robots, andused the lack of synchronization to detect faults.

Researchers have also focused on understanding the basis for the evolution of cooperation [27], andadditional insight into how cooperation might arise has been uncovered through both digital androbotics experiments [13, 24]. Many approaches to the evolution of cooperation in nonbiological sys-tems involve multilevel selection, the theory that selection acts on both individuals and the groups to whichan individual belongs. Although the role of multilevel selection is somewhat controversial in biology,Waibel and colleagues [37] have demonstrated its effectiveness in evolving cooperative behaviors forcomputational systems. Other researchers have used forms of multilevel selection to improve the run-time performance of multi-objective evolutionary algorithms [14], and even to diagnose malignancy incancer [39]. Additionally, genetic programming has been used to design communication protocols forautonomic systems, where individuals in the population were different protocols [38]. Evolutionarycomputation has also been applied to the scheduling of collective communication among multipleCPU cores on a parallel computer, including evolution of known scheduling algorithms [17].

A recent study of the evolution of synchronization for robotic swarms was conducted by Trianni andNolfi [36]. In that study, a neural network controller was evolved to synchronize the movements ofrobots within an enclosed arena. The robots were able to sense a symmetric gradient painted on thefloor of the arena, and could communicate globally via a single audible tone. Our objective in the studydescribed herein was different, in that we sought to determine whether (and how) populations of digitalorganisms can evolve both synchronization and desynchronization behaviors when provided withprimitives akin to those of fireflies. Specifically, the organisms had no environmental cues and were onlycapable of local communication.

3 Avida Background

The Avida digital evolution platform has previously been used to investigate fundamental aspects of theevolutionary process, including biocomplexity [23], genetic organization [26], specialization [6, 30], andphenotypic plasticity [8]. To enable the exploration of an even wider range of evolved behaviors, wehave extended Avida with an assortment of primitives enabling organisms to communicate with oneanother, move about their environment, and sense their surroundings. Our recent studies have demon-strated evolution of a variety of collective behaviors, including distributed consensus [19], quorum sens-ing [4], and network construction [22].

3.1 Digital OrganismsFigure 1 depicts an Avida population and the structure of an individual organism. Each digital organismcomprises a circular list of instructions (its genome) and a virtual CPU, and exists in a common virtualenvironment. Within this environment, organisms execute the instructions in their genomes, and theparticular instructions that are executed determine the organismʼs behavior (its phenotype). Different AvidaCPU architectures have been implemented and used in various studies [29]. The architecture used in thisstudy contains a circular list of three general-purpose registers {AX,BX,CX}, two general-purpose stacks{GS, LS}, and four special-purpose heads. Heads may be thought of as pointers into the organismʼs gen-ome and are similar to a traditional program counter and stack pointer. The instruction head points tothe next instruction to be executed. The flow-control head points to a location in the genome to whichthe instruction head may be moved upon execution of certain instructions; it is similar to a goto labelin the C programming language, but may be dynamically changed during execution. The read head and

Artificial Life Volume 17, Number 1 3


write head are primarily used during replication, and enable the organism to read and write instructionswithin its genome.

Instructions within an organismʼs genome are similar in appearance and functionality to traditionalassembly language instructions. These instructions enable an organism to perform simple mathematicaloperations, such as addition, multiplication, and bit shifts; to manipulate the position of heads within itsgenome; to sense and change properties of the environment; and to communicate with neighboring or-ganisms. Certain instructions also enable organisms to replicate, subject to mutation, and thus spreadthroughout the population. Instructions within Avida can have different costs (in terms of virtual CPUcycles) associated with them. For example, a simple addition instruction may cost only one cycle, whilebroadcasting a message may cost 20 cycles. A key property of Avidaʼs instruction set that differs fromtraditional computer languages is that it is not possible to construct a syntactically incorrect genome in AVIDA;that is, all possible genomes are runnable. Hence, while random mutations may produce many genomesthat do not perform any meaningful computation, their instruction sequences are always executable.

A sample fragment of an organismʼs genome and its corresponding pseudocode are shown inFigure 2. This series of four instructions causes the organism to issue a flash if it had received a flashat any point during its lifetime. Two of the instructions in this fragment, flash-info and flash, were im-plemented for this study and are described in more detail in Section 4.

As shown in Figure 1, each organism in Avida lives in a cell located in a fixed location within a spa-tial environment. Each cell can contain at most one organism; organisms cannot live outside ofcells. The topology of the environment determines the neighborhood of each cell, and is user-defined.For example, the environment topology may be configured as a grid, as a torus, or as a well-mixedenvironment, where all cells are neighbors of each other. The particular environmental topology canaffect characteristics of certain virtual CPU instructions. For example, an instruction that broadcastsa message in the callerʼs neighborhood will send to different organisms depending on the environ-mental topology.

Figure 2. Sample fragment of an organismʼs genome, and corresponding pseudocode.

Figure 1. An Avida population containing multiple genomes (bottom), and the structure of an individual organism (top).



During its replication cycle, an organism executes instructions that copy its genome to its offspring.As the genome is copied, mutations may be introduced according to predefined probabilities. Thesemutations may take the form of a replacement (substitution of a random instruction), an insertion(inserting an additional, random instruction into the offspringʼs genome), or a deletion (removingthe copied instruction from the offspringʼs genome).

3.2 Multilevel SelectionNatural selection is the differential survival and reproduction of organisms within their environment [9],while multilevel selection is the theory that the survival of the individual is linked to the survival of itsgroup. These groups may be defined in many different ways, for example, by a common trait (a traitgroup), shared ancestry (clade selection), membership in the same species (species selection), or theinteractions between related individuals (kin selection). Although somewhat controversial in biology,multilevel selection has been used to explain the evolution of altruism, where an individual will sacrificeitself for the benefit of the group, and the evolution of social behavior, particularly in populations ofsocial insects, such as ant and bee colonies [34]. In engineering, multilevel selection has been shown tobe an effective means to produce cooperative behaviors for multi-agent systems [13, 19, 21, 22, 37].

In Avida, selection can take place at both individual and group levels. At the individual level, organ-isms compete with each other for space in their environment and are responsible for their own replica-tion (i.e., the instructions for replication must be present in each organismʼs genome). Althoughindividual selection is an important part of most Avida research, for this study we use group selectiononly. There are two components to group selection within Avida. First, the population of digital organ-isms is divided into distinct subpopulations, called demes. Second, these demes replicate and competewith each other for space and other resources. Figure 3 depicts an environment that has been subdi-vided into sixteen demes.

For this study, we used CompeteDemes, a framework within Avida that enables the periodic replica-tion and competition among demes. During the execution of an Avida trial, the CompeteDemes frame-work periodically calculates the fitness of each deme via a user-defined fitness function. This fitnessfunction takes as input a single deme (and all the organisms within), and produces the fitness of thatdeme (a floating-point number) as output. For example, one may define a fitness function based on thedegree of synchronization exhibited by the organisms within a deme. Using the resulting array of fitnessvalues, the CompeteDemes framework then performs fitness-proportional selection, which preferentiallyreplicates those demes with higher fitness, and replaces those demes with lower fitness. The number ofdemes that reproduce during each competition thus depends on the distribution of fitness values: If alldemes have approximately the same fitness, each will advance to the next generation (on average). How-ever, if a single deme has much higher fitness than others, it is likely to replace many other less fit demes.When using a fitness function that rewards for synchronization, the CompeteDemes framework will thenpreferentially replicate those demes that are more synchronized than others, resulting in a populationthat evolves toward higher levels of synchronization.

Figure 3. Depiction of an Avida population of 16 demes, with 64 organisms per deme. Demes are isolated subpopulations,each capable of replication. When a deme replicates, it replaces a randomly selected target deme.



3.3 Germline ReplicationAs with selection, replication in Avida can also occur at two different levels, the individual and thedeme. When using CompeteDemes, each deme is initialized with a single organism, called the seed, atthe start of every competition period. The seed may be an organism that is transferred from a sourcedeme to a target during deme replication, or it may be created from a demeʼs germline, which providesa single common genetic ancestry for all organisms within a deme. In animals, genetic material istransferred from parent to offspring along the germline [11]. Germ cells are distinct from somatic cells,which form the body of an organism. Mutations to an organismʼs germline can be passed to itsoffspring, while mutations to an organismʼs somatic cells will typically affect only the host.

In Avida, the germline is a genome that is associated with a deme, rather than with an individual.Although individuals within a deme may still self-replicate, mutations occur only along the germline, andthen only during deme replication. When a deme replicates (the arrows in Figure 3), the germline for thesource deme is copied (subject to mutation) to the target deme, and the seed for the target deme isinserted. In an earlier study, we found that using a digital germline was necessary to evolve organismsthat cooperated to construct communication networks [22]. Due to the high degree of coordinationrequired for synchronization, we also used digital germlines in the studies presented here. Germlinereplication has the side effect of homogenizing the inhabitants of each deme, a technique that has beenshown to be effective in evolutionary robotics [13].

Figure 4 illustrates GermlineReplication, the process used to replicate demes that use a germline. Be-ginning from an ancestral germline g0, the parent deme is seeded with an organism generated from thelatest germ. During the course of the experiment, the somatic organisms within this deme replicate,compete for resources, and may experience mutations. Once a deme replication is triggered, all organ-isms within the parent deme are killed, and the parent is reseeded from its germline, g0. The latest germfrom g0 is then mutated, producing a new germline, g1. Any organisms living in the offspring demeare then killed, and the new germ from g1 is used to seed the offspring deme. When used with theCompeteDemes framework, the parent demes are selected via fitness-proportional selection, while theoffspring demes are those that were not selected to replicate.

4 Methods

Typically an Avida user will configure the environment in which the digital organisms live and define theselective pressures that act upon the population. Once configured, a set of independent Avida trials(typically 10–30) is conducted, and the results are analyzed. Multiple Avida trials are used to allowfor the stochastic nature of evolution; although a single result may be sufficient to evolve a particularbehavior, multiple trials improve statistical accuracy. In the remainder of this section, we describe theAvida configuration we used to evolve synchronization and desynchronization.

4.1 InstructionsWe implemented six new virtual CPU instructions for this study, which are summarized in Table 1. Eachof these instructions either was inspired by observations of fireflies (e.g., flash) or was implementedto provide digital organisms with information not previously available (e.g., flash-info). When an

Figure 4. Example of deme replication using a digital germline.



organism executes the flash instruction, a flash message is broadcast to each of its neighbors, and infor-mation about this message is automatically stored in a special register that can be accessed only via theif-recvd-flash, flash-info, and flash-info-b instructions. The if-recvd-flash instruction conditionally executes thenext instruction in the organismʼs genome if the caller had received a flashmessage since its birth or the lastexecution of the hard-reset instruction (described below). The flash-info instruction is used to determinewhether the organism has ever received a flash, and if so, sets a register to the elapsed time (numberof virtual CPU cycles) since the last flash message had been received. Similarly, the flash-info-b instructionis used to determine whether the organism has ever received a flash, but does not store the time at whichthat flash was received. The hard-reset instruction resets the organism to its initial state (as if the organismwere just born). Finally, the get-cycles instruction retrieves the number of virtual CPU cycles since theorganism was born or last issued a hard-reset, whichever is most recent. We emphasize that these instruc-tions are simply available to the evolutionary process; whether they are incorporated into a digital organ-ismʼs genome is a function of mutation and selection.

4.2 ConfigurationWe configured Avida to use the CompeteDemes process outlined in Section 3.2, with specific configurationvalues summarized in Table 2. We used 400 demes, each with a 5 × 5 torus topology, and each deme wasconfigured to use a germline. The small size of each deme was selected primarily for practical (namely,computation time) reasons. We used the default values for the different mutation rates, summarized inTable 2. To improve data granularity, we also adjusted the time slice down from a default of 30 instruc-tions per update to five instructions per update, where an update is the standard unit of experiment timein Avida. An update corresponds to each organism receiving, on average, a number of virtual CPU cyclesequal to the time slice. We performed 30 trials of each different experiment, except where noted.

Table 1. New instructions implemented for this study. All instructions are equally likely to be selected as targets for mutation.

Artificial Life Volum

Instruction

e 17, Number 1

Description

flash
Broadcasts a flash message to callerʼsneighbors, with a configurable loss rate.
if-recvd-flash
If the caller has received a flash from anyof its neighbors, execute the subsequentinstruction. Otherwise, skip thesubsequent instruction.
flash-info
If the caller has ever received a flash, setBX to 1 and CX to the number of cyclessince that flash was received. Otherwise,set BX and CX to 0.
flash-info-b
If the caller has ever received a flash, setBX to 1; do not modify CX.
hard-reset
Reset the state of the virtual CPU to theorganismʼs birth state. All registers arezeroed out, and heads are reset, includingthe flash timer and cycle counter.
get-cycles
Set BX to the number of virtual CPU cyclessince either the organism was born orthe last time hard-reset was called,whichever is most recent.
7


In order to control the initial conditions for each deme, we configured Avida to disallow self-replication of individual organisms. For the synchronization experiments we then configured an event,RandomInjectGerm, to probabilistically (50%) insert an organismwith the germlineʼs genome into an emptycell in each deme every update, thus ensuring that the organisms in a deme were not initialized to a syn-chronized state. Similarly, in the desynchronization experiments we initialized each deme to be comple-tely filled with synchronized organisms built from the germlineʼs genome. These different forms ofinitialization ensured that demes were not synchronized (or desynchronized) due to artifacts of Avida.

As the final step of Avida configuration, we defined the default ancestor, which is the starting organismfor each deme within all Avida trials. Here, we used an organism that contained 99 nop-C instructions(the nop-A, nop-B, and nop-C instructions perform no computation, and do not change the state of thevirtual CPU), followed by single flash as the final instruction. All experiments used this organism as theirdefault ancestor. The presence of a large number of nop-C instructions in the default ancestoris common in Avida experiments, and provides evolution with a “blank tape” for mutating differentinstructions into the genome. We emphasize that although the default ancestor contains only 100 in-structions in its genome, not only can mutations increase and decrease the genome size, but the genomeitself is circular; once the organism executes the final instruction, execution flow wraps around to thebeginning of the genome.

5 Synchronization Study

In this study, we focused on heartbeat synchronization [2] and evolved digital organisms that can syn-chronize flashes to within a single update, or five virtual CPU cycles, of each other. We initialized each

Table 2. AVIDA configurations used for the synchronization and desynchronization experiments described in this study.

8

Configuration
Value
Trials per experiment
30, except where noted
Maximum population size
10,000
Number of demes
400, each 5 × 5
Environment topology
Torus
Copy mutation rate
0.0075 (per instruction)
Insertion mutation rate
0.05 (per replication)
Deletion mutation rate
0.05 (per replication)
Time slice
Five instructions per update,constant-time scheduler
CompeteDemes
Compete all demes usingfitness-proportional selection(periodic, every 200 updates)
RandomInjectGerm
50% probability to inject a copyof the latest germ into an emptycell in each deme (periodic,every update)
Artificial Life Volume 17, Number 1


deme to be in a state of desynchronization by starting from a single organism, and inserted exactly onenew organism with a 50% probability each update. We configured the CompeteDemes framework tocompete all 400 demes with each other every 200 updates, and the fitness function defined belowwas based on the execution timing of the flash instruction within each deme. Within each deme wemonitored both the number of unique organisms that executed the flash instruction, as well as the ab-solute number of flash instructions that were issued.

Fitness function We defined a fitness function that rewarded demes whose constituent organ-isms each executed the flash instruction, and further rewarded demes for increasing the differencebetween the maximum number of flashes and the mean number of flashes. Specifically, the fitnessfunction used for synchronization was defined as

F ¼ 1þ U if U < S1þ U þ ðmaxðCÞ − meanðCÞÞ2 if U ¼ S

�ð1Þ

where F is the fitness of the deme, U is the number of unique organisms that issued a flash, S is thenumber of organisms in the deme (25 for these experiments), max(C ) is the maximum number offlashes during any single update, and mean(C ) is the mean number of flashes per update. This fitnessfunction thus rewards demes whose constituent organisms issue the flash instruction, and if all organ-isms in the deme issue a flash, further rewards the deme for increasing degrees of flash synchronization.Each of max(C ), mean(C ), and U was calculated from the final 50 updates of each CompeteDemes pe-riod to allow organisms a time (the first 150 updates) during which they may issue flash instructionswithout adversely affecting the demeʼs fitness. This approach grants evolution a great degree of flex-ibility in the kinds of strategies that may evolve. Maximum fitness is achieved when all organisms flashexactly once, at approximately the same time, during the final 50 updates of a competition period. Wenote that synchronization did not evolve when fitness was calculated over the entire 200-update period.

Treatments To better understand which factors influenced the evolution of synchronization, weconducted three different treatments of the synchronization experiment. The flash-info treatment used a20-cycle cost for execution of the flash instruction (all others required only a single cycle). The zero-costtreatment modified the flash-info treatment by reducing the cost of executing the flash instruction toone cycle, making the flash instruction no more expensive to execute than any other instruction in thegenome. Both the flash-info and zero-cost treatments were allowed to use the flash-info instruction. Fi-nally, the no-timer treatment modified the flash-info treatment by replacing the flash-info instruction withflash-info-b, which prevented organisms from accessing timing information related to the reception offlash messages.

5.1 Experimental ResultsFigure 5 plots the grand mean number of coincident flashes over all 30 Avida trials, for each of thesethree different treatments. Here we see that after 100,000 updates, the mean difference between max-imum and mean flash counts approaches 20 in all three treatments. In other words, after 500 genera-tions (100,000 updates; 200 updates per CompeteDemes period), organisms within the average deme ineach treatment have synchronized with each other, and we see that this behavior does not depend oninstruction cost or knowledge of the exact timing of message reception. We emphasize that this figureshows the average results. A number of demes achieved perfect synchronization; an example is shownlater in Figure 8. We note that the final results of these treatments are not statistically significantly dif-ferent from each other (Kruskal-Wallis multiple comparison, test for significance performed at the p <.05 level; SD at final update: 2.09, 2.33, 2.27).

Figure 6 plots the grand minimum number of coincident flashes over all trials (i.e., the least fit ofall demes). Although this figure does little for evaluating the performance of the evolved solutions, itdoes grant insight into the evolutionary process that gave rise to synchronization. Specifically, here wesee that the zero-cost treatment has the highest fitness until approximately update 95,000, when the



performance of the flash-info treatment improves. This result indicates that having an instruction costfor issuing a flash hinders the evolvability of synchronization. We also see that the no-timer treatmentperformed worst for approximately 70,000 updates, indicating that the availability of message timinginformation improved the evolvability of synchronization.

To determine whether the evolution of synchronization is resilient to lost flash messages, we nextexamined the effect of flash loss rate on the evolution of synchronization. Specifically, we varied theflash loss rate from 0% to 80%, where we defined a flash loss as a flash message that is lost in sending,that is, it is not received by any of its neighbors. Each of these treatments used a unit-cost flash instruc-tion. Figure 7 plots the number of coincident flashes versus time, averaged over each deme in 10 trials.We iterated over different flash loss rates from 0% to 80% under the synchronization treatment. Herewe see that higher loss rates inhibit the evolution of synchronization behavior, although we note thatevolution was still able to discover solutions comparable to the 0% loss rate case at loss rates up to 20%.Statistically significant differences of the final results were found between the {0%, 10%, 20%} and{40%, 80%} treatment groups, but no statistically significant differences were found within thesegroups (Kruskal-Wallis multiple comparison, test for significance performed at the p < .01 level).

Figure 8 is a detail of the behavior of a single deme containing 25 copies of the dominant (that is, mostcommon) evolved genome from the end of the flash-info trials. Each point in this plot represents theexecution of the flash instruction by a particular organism. Here we see run-time synchronization, wherethe 25 individuals within the deme have evolved to synchronize their flashes at an identical phase and

Figure 6. Worst-case performance of synchronization trials, showing that the zero-cost treatment is most able to evolvesynchronization, and that the availability of timing information improves evolvability.

Figure 5. Evolution of synchronization using a fitness function that rewards for synchronization in execution of theflash instruction, under three different treatments.



periodwithin 200 updates. Of note is that evolution has solved two problems: the period that organismswillsynchronize to, and the behavior that brings them into synchronization. The organisms within this demehave synchronized to a period of 19 updates. Different demes within this same trial, as well as those indifferent trials, may well synchronize to a different period. Avideo showing the execution of the genomesof the organisms in this deme is available at the following URL: www.cse.msu.edu/thinktank/firefly.

5.2 Genome AnalysisThe Avida TestCPU enables the analysis of a single organism in an isolated environment. To analyzethe synchronization and desynchronization behavior of an Avida genome, we extended the TestCPU

to support the artificial delivery of a flash message to an organism under test. We then traced theorganismʼs execution flow that resulted from receiving a flash at different times, and monitored itsresponse in terms of flash execution.

Herewe analyze the dominant genome from the end of trial 1 under the flash-info treatment. Trial 1wasone of the highest-performing trials in the synchronization experiment, as determined by theCompeteDemes fitness function. This treatment used a 20-cycle cost for issuing the flash instruction, andused flash-info, which provided the organism with timing information of the last message received.

Figure 9 plots the response of the dominant to receiving a flash at various times during its life. Ateach point t along the horizontal axis, we initiated a new test of the dominant, and artificially sent a


Figure 7. Evolution of synchronization over different uniform flash loss rates.

Figure 8. Detail of evolved synchronization behavior. Organisms initially flash asynchronously, but gradually synchronizewith each other.


single flash message to the organism once it had executed t virtual CPU cycles. We then monitoredthe response of the organism from each test for 2000 cycles, and plotted a point for each cycle spentexecuting the flash instruction. For example, in Figure 9 at time 100, we see a series of horizontalbands every 110 cycles. This indicates that if the organism receives a single flash after it has executed100 instructions, it will respond by flashing every 110 cycles. The gaps from time 1 to 19 and neartimes 50, 110, and 150 are artifacts of the organism executing the hard-reset instruction.

Figure 10 zooms in on the first 180 cycles of the organismʼs response to receiving flash messages atdifferent times. Here we see evidence of the strategy employed for synchronization. First, the horizontalband extending from time 19 to 200 indicates that, regardless of flash reception, organisms will issue aflash that will complete by their 66th virtual CPU cycle. Region C in Figure 10, extending from time 108to 200, shows that the organism has the same response to receiving a flash during this interval, indicatingthat the organism is not making any phase or frequency adjustments to its flash execution. Region A, onthe other hand, shows a quicker execution of flash than region C, indicating that the reception of amessage in this region causes a frequency advance of the receiver. Finally, region B shows a slowerresponse to flash messages than region A, indicating a region of frequency delay. This combinationof steady-state and adaptive frequency operation is similar to the models of biological synchronizationfrom Mirollo, Strogatz, and Ermentrout [12, 25].

Figure 10. Detailed synchronization response of an organism to receiving flash messages at different times. Region C is asteady-state response, while regions A and B are frequency advance and frequency delay responses, respectively.

Figure 9. Synchronization response of the dominant genome to receiving flash messages at different times.



Figure 11 depicts the genome responsible for the synchronization behavior shown in Figure 8, andhighlights the primary instructions. Following birth, execution flow of this genome proceeds nearlylinearly for 65 cycles, after which point it begins periodic execution of the flash instruction. However,if a flash is received, its execution flow dramatically changes. Specifically, it will use a combination ofhard-reset and jmp-head, an instruction that enables numeric manipulation of the organismʼs instructionpointer, to either advance or delay the time at which flashwill be executed. Interestingly, the organismwilluse its age as returned by get-cycles and the elapsed time since a message had been received from flash-info

to modify the destination for the jmp-head instruction, providing an additional means of frequency ad-justment. Finally, we note the large number of instructions within this genome that have no discerniblepurpose. These instructions are most likely neutral mutations, or mutations that do not adversely affectfitness. We note, however, that their presence plays a role in determining the native flash period, as theystill require virtual CPU cycles to be spent on their execution.

6 Desynchronization Study

In this study, our goal is to discover behaviors that bring digital organisms out of phase with respectto each other, similarly to round-robin scheduling [33]. As much as possible, the desynchronizationexperiment mirrors our study of synchronization—its Avida configuration and experiment treatmentdesign are, in fact, identical. Only two factors are different: First, we initialized each deme to be in a stateof synchronization (that is, all organisms are born at the same time and in the same state), and second,we altered the fitness function used. That the opposite temporal behavior can be evolved with onlythese changes illustrates the flexibility of Avida in producing different behaviors specified by simplefitness functions.

Fitness Function We defined a fitness function that rewards demes whose constituent or-ganisms each execute the flash instruction, and then further rewards demes for decreasing the num-ber of organisms that flash during the same update. Specifically, the fitness function used fordesynchronization was defined as

F ¼ 1þ U if U < S1þ U þ ðS − maxðCÞÞ2 if U ¼ S

�ð2Þ

where again, F is the fitness of the deme, U is the number of unique organisms that issued a flash

instruction, S is the number of organisms in the deme, and max(C ) is the maximum number offlashes during any single update; both max(C ) and U were calculated from the final 50 updatesof each CompeteDemes period. This fitness function thus rewards demes whose constituent organ-isms issue the flash instruction, and if all organisms flash, further rewards the deme for the level offlash desynchronization. Maximum fitness is achieved when all organisms in a deme flash at differenttimes during the final 50 updates of a competition period.

6.1 Experimental ResultsAs with synchronization, we conducted three different desynchronization treatments, flash-info, zero-cost,and no-timer. As before, these treatments used a 20-cycle cost for the flash instruction, used a single-cyclecost for flash, and replaced the flash-info instruction with flash-info-b, respectively.

Figure 12 plots the grand mean number of coincident flashes versus updates averaged over eachdeme in all 30 trials, from each of these treatments. Here we see that over the course of the experiment,the level of desynchronization, measured as worst-case coincident flashes, dropped from 25 (entiredeme flashing in unison) to an average of approximately 6.1 after 100,000 updates. (In the best-performing trial, this value dropped to approximately 5.0.) We note that the final results of these treat-ments are not statistically significantly different from each other (Kruskal-Wallis multiple comparison,test for significance performed at the p < .05 level; SD at final update: 1.10, 1.26, 1.20), which indicates




-
Figure 11. Depiction of the dominant genome for synchronization. Labels A, B, and C refer to destinations of the jmphead instruction that correspond to frequency-advance, frequency-delay, and steady-state regions from Figure 10.


that desynchronization does not depend on instruction cost or knowledge of the exact timing of mes-sage reception.

As with the study of synchronization, we also examined the effect of a flash loss rate on the evolutionof desynchronization. We again varied the loss rate from 0% to 80%, and used a unit-cost flash instruc-tion. Figure 13 plots the number of coincident flashes versus time averaged over each deme in 10 trials,iterated over different flash loss rates from 0% to 80% loss under the desynchronization treatment.Interestingly, this figure shows that desynchronization evolves more quickly under the 80% loss ratethan under the 0% loss rate, which shows that evolution can take advantage of characteristics of theenvironment (in this case, random flash loss) to solve a particular problem. Here, not only does therandom loss of a flash decrease the likelihood of coincident flashes, but these losses also serve as asource of randomness, which assists individual organisms in spreading their flashes throughout phasespace. No statistically significant differences of the final results were found between these treatments(Kruskal-Wallis multiple comparison, test for significance performed at the p < .05 level).

Figure 14 is a detail of the behavior of a single deme containing 25 copies of the dominant ge-nome from the end of one of the best-performing Avida flash-info trials. Each point again repre-sents the execution of the flash instruction by a particular organism. Here, however, we see that allorganisms initially emit a single flash in unison near update 20, and thereafter progressively decreasethe level of synchronization in the deme. We note that this evolved behavior is not perfectly de-synchronized; there are four coincident flashes near update 175.

Figure 13. Evolution of desynchronization over different uniform flash loss rates.

Figure 12. Evolution of desynchronization using a fitness function that rewards for desynchronization in execution of theflash instruction, under three different treatments.



6.2 Genome AnalysisHere we analyze the dominant genome from the end of trial 14 of the flash-info treatment. Trial 14 wasone of the highest-performing trials in the desynchronization experiment. As with synchronization,this treatment used a 20-cycle cost for issuing the flash instruction, and used flash-info to provide theorganism with timing information on the last message received.

Figure 15 plots the response of the dominant to receiving a flash at various times during its life,similarly to Figure 9. At each point t along the horizontal axis, we initiated a new test of the dominant,and artificially sent a single flash message to the organism once it had executed t virtual CPU cycles. Wethen monitored the response of the organism from each test for 2000 cycles, and plotted a point foreach cycle spent executing the flash instruction. As before, the gap from time 13 to 19 is an artifact of theorganism executing the hard-reset instruction.

Figure 16 zooms in on the first 200 cycles of the organismʼs response to receiving flash messagesat different times, enabling us to characterize the strategy employed for desynchronization. First, thehorizontal band extending from time 57 to 199 indicates that, regardless of flash reception, organismswill issue a flash that will complete by their 97th virtual CPU cycle. Regions A and C exhibit thesame frequency-advance behavior as with the synchronization dominant, while regions B and D exhibitsteady-state behavior, although at slightly different frequencies.

Figure 15. Desynchronization response of the dominant genome to receiving flash messages at a different times.

Figure 14. Detail of evolved desynchronization behavior. Organisms initially flash in unison, but gradually increase thelevel of desynchronization present.



Figure 17 depicts the instructions present in the dominant genome, and shows the primary instruc-tions that are responsible for the behavior in Figure 14. There is a single flash instruction present in thisgenome. Upon birth, and when it does not receive a flash message, this organism executes the first 91instructions approximately in order, completing its own flash at cycle 110. As with the synchronizationdominant, if this organism receives a flash at any point during its lifetime, its execution flow changes.Here, however, rather than using the age of the organism and the elapsed time since the last flash mes-sage was received, this organism modifies the position of its write head (a pointer into its genome,similar to a stack pointer ) in response to received flash messages to determine the next point intime at which to execute the flash instruction. Similar to the engineered algorithms in [33], this genomeattempts to iteratively adjust the organismʼs phase and frequency until it is desynchronized. Here, how-ever, the evolved strategy is dependent on local information only—it requires neither access to its neigh-bors in phase space, nor sending flashes globally.

7 Conclusion

We have shown that digital evolution can produce cooperative behaviors for synchronization and de-synchronization when provided with primitives based on biological fireflies and purely local informa-tion. The evolved solutions utilized an adaptive frequency strategy, similar to models of firefly behavior,that altered their control flow based on a combination of sensed information and the organismʼs age. Wehave also shown that relatively complex cooperative behaviors can be evolved via simple fitness func-tions, that both synchronization and desynchronization can be evolved in Avida by a straightforwardchange in fitness functions, and that the evolution of these behaviors is robust to message loss. Theresults demonstrate that digital evolution shows promise as a tool for constructing self-organizing sys-tems. We also note that although we have used only simple fitness functions in this work, the criteriaused to evaluate solutions can be more complex, and can include both intra-Avida evaluation as used inthis study, and external tools, for example, by cross-compiling an Avida organism to a network simulatorfor evaluation.

8 Further Information

Further information on evolving adaptive and cooperative behavior can be found at www.cse.msu.edu/thinktank. The extensions to Avida that are described in this study have been made available as part ofthe Avida software distribution. Source code and other information is available at devolab.msu.edu.

Figure 16. Detailed desynchronization response of an organism to receiving flash messages at different times. Regions Aand C are frequency-advance responses, while regions B and D are steady-state responses.




Figure 17. Depiction of the dominant genome for desynchronization.


AcknowledgmentsWe thank the anonymous reviewers for their many thoughtful suggestions on how to improve themanuscript. We also thank members of the Digital Evolution Laboratory and the Software Engineeringand Network Systems Laboratory for their contributions to this work, including early feedback. Thisresearch was supported in part by National Science Foundation grants CNS-0915855, CCF-0820220,and CNS-0751155; by U.S. Army grantW911NF-08-1-0495; and by aQuality Fund grant fromMichiganState University.

References1. Adami, C., Ofria, C., & Collier, T. C. (2000). Evolution of biological complexity. Proceedings of the National

Academy of Sciences (USA), 97, 4463–4468.

2. Babaoglu, O., Binci, T., Jelasity, M., & Montresor, A. (2007). Firefly-inspired heartbeat synchronizationin overlay networks. In Proceedings of Self-Adaptive and Self-Organizing Systems (SASO) (pp. 77–86).

3. Backwell, P., Jennions, M., Passmore, N., & Christy, J. (1998). Synchronized courtship in fiddler crabs.Nature, 391(6662), 31–32.

4. Beckmann, B. E., & McKinley, P. K. (2009). Evolving quorum sensing in digital organisms. In Proceedingsof the Genetic and Evolutionary Computation Conference (GECCO) (pp. 97–104).

5. Buck, J. (1998). Synchronous rhythmic flashing of fireflies, II. The Quarterly Review of Biology, 63(3), 265–289.

6. Chow, S. S., Wilke, C. O., Ofria, C., Lenski, R. E., & Adami, C. (2004). Adaptive radiation fromresource competition in digital organisms. Science, 305, 84–86.

7. Christensen, A. L., OʼGrady, R., & Dorigo, M. (2009). From fireflies to fault-tolerant swarms ofrobots. IEEE Transactions on Evolutionary Computation (TEC), 13(4), 754–766.

8. Clune, J., Ofria, C., & Pennock, R. T. (2007). Investigating the emergence of phenotypic plasticityin evolving digital organisms. In Proceedings of the 9th European Conference on Artificial Life (ECAL)(pp. 74–83).

9. Conner, J. K., & Hartl, D. L. (2004). A primer of ecological genetics. Sunderland, MA: Sinauer.

10. Davey, M. E., & OʼToole, G. A. (2000). Microbial biofilms: From ecology to molecular genetics.Microbiology and Molecular Biology Reviews, 64(4), 847–867.

11. Eddy, E. (1996). The germ line and development. Developmental Genetics, 19, 287–289.

12. Ermentrout, B. (1991). An adaptive model for synchrony in the firefly Pteroptyx malaccae. Journal ofMathematical Biology, 29(6), 571–585.

13. Floreano, D., Mitri, S., Magnenat, S., & Keller, L. (2007). Evolutionary conditions for the emergenceof communication in robots. Current Biology, 17(6), 514–519.

14. Giel, O., & Lehre, P. K. (2006). On the effect of populations in evolutionary multi-objective optimization.In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO) (pp. 651–658).

15. Grimaldi, D., & Engel, M. S. (2005). Evolution of the insects. Cambridge, UK: Cambridge University Press.

16. Holldobler, B., & Wilson, E. O. (1990). The ants. Boston: Harvard University Press.

17. Jaros, J., Ohlidal, M., & Dvorak, V. (2007). An evolutionary approach to collective communicationscheduling. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO) (pp. 2037–2044).

18. Jelasity, M., & Babaoglu, O. (2005). T-man: Gossip-based overlay topology management. In Proceedingsof the Workshop on Engineering Self-Organising Applications (ESOA) (pp. 1–15).

19. Knoester, D. B., & McKinley, P. K. (2009). Evolution of probabilistic consensus in digital organisms.In Proceedings of Self-Adaptive and Self-Organizing Systems (SASO) (pp. 223–232).

20. Knoester, D. B., & McKinley, P. K. (2009). Evolving virtual fireflies. In Proceedings of the European Conferenceon Artificial Life (ECAL). Berlin: Springer.

21. Knoester, D. B., McKinley, P. K., & Ofria, C. (2007). Using group selection to evolve leadership inpopulations of self-replicating digital organisms. In Proceedings of the Genetic and Evolutionary ComputationConference (GECCO) (pp. 293–300).



22. Knoester, D. B., McKinley, P. K., & Ofria, C. (2008). Cooperative network construction using digitalgermlines. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO) (pp. 217–224).

23. Lenski, R. E., Ofria, C., Pennock, R. T., & Adami, C. (2003). The evolutionary origin of complexfeatures. Nature, 423, 139–144.

24. Marocco, D., & Nolfi, S. (2006). Self-organization of communication in evolving robots. In Proceedingsof the Conference on Artificial Life (ALIFE) (pp. 178–184).

25. Mirollo, R. E., & Strogatz, S. H. (1990). Synchronization of pulse-coupled biological oscillators.SIAM Journal on Applied Mathematics, 50(6), 1645–1662.

26. Misevic, D., Ofria, C., & Lenski, R. E. (2006). Sexual reproduction reshapes the genetic architectureof digital organisms. Proceedings of the Royal Society B, 273, 457–464.

27. Nowak, M A. (2006). Five rules for the evolution of cooperation. Science, 314(5805), 1560–1563.

28. Ofria, C., & Adami, C. (1999). Evolution of genetic organization in digital organisms. In Proceedingsof DIMACS Workshop on Evolution as Computation.

29. Ofria, C., & Wilke, C. O. (2004). Avida: A software platform for research in computational evolutionarybiology. Journal of Artificial Life, 10, 191–229.

30. Ostrowski, E., Ofria, C., & Lenski, R. E. (2007). Ecological specialization and adaptive decay in digitalorganisms. The American Naturalist, 169, E1–E20.

31. Oullier, O., de Guzman, G. C., Jantzen, K. J., Lagarde, J., & Scott Kelso, J. A. (2008). Social coordinationdynamics: Measuring human bonding. Social Neuroscience, 3(2).

32. Palva, J. M., Palva, S., & Kaila, K. (2005). Phase synchrony among neuronal oscillations in the humancortex. Journal of Neuroscience, 25(15).

33. Patel, A., Degesys, J., & Nagpal, R. (2007). Desynchronization: The theory of self-organizing algorithmsfor round-robin scheduling. In Proceedings of Self-Adaptive and Self-Organizing Systems (SASO) (pp. 87–96).

34. Seeley, T. D. (1997). Honey bee colonies are group-level adaptive units. The American Naturalist, 150,S22–S41.

35. Southwick, E. E., & Moritz, R. F. A. (1987). Social synchronization of circadian rhythms of metabolismin honeybees (Apis mellifera). Physiological Entomology, 12(2), 209–212.

36. Trianni, V., & Nolfi, S. (2009). Self-organizing sync in a robotic swarm: A dynamical system view.IEEE Transactions on Evolutionary Computation, 13(4), 722–741.

37. Waibel, M., Keller, L., & Floreano, D. (2009). Genetic team composition and level of selection in theevolution of cooperation. IEEE Transactions on Evolutionary Computation, 13(3).

38. Yamamoto, L., & Tschudin, C. F. (2005). Experiments on the automatic evolution of protocols usinggenetic programming. In Proceedings of the IFIP Workshop on Autonomic Communication (WAC).

39. Zechman, E. M., & Ranjithan, S. R. (2005). Multipopulation cooperative coevolutionary programming(MCCP) to enhance design innovation. In Proceedings of the Genetic and Evolutionary Computation Conference(GECCO) (pp. 1641–1648).


evolution of synchronization and desynchronization in digital...

Documents