evolution as a tool for understanding and designing collaborative systems
DESCRIPTION
Keynote talk by Wilfried Elmenreich at PRO-VE 2011:Self-organizing phenomena can be found in many social systems, either forcing collaboration or destroying it. Typically, these properties have not been designed by a central ruler but evolved over time. While it is straightforward to find examples in many social systems, finding the appropriate interaction rules to design such systems from scratch is difficult due to the unpredictable or counterintuitive nature of such emergent and complex systems. Therefore, we propose evolutionary models to examine and extrapolate the effect of particular collaboration rules. Evolution, in this context, does not replace the work of analyzing complex social systems, but complements existing techniques of simulation, modeling, and game theory in order to lead for a new understanding of interrelations in collaborative systems.TRANSCRIPT
Evolution as a Tool for Understanding and Designing Collaborative SystemsWilfried ElmenreichMobile Systems Group/Lakeside LabsInstitute for Networked and Embedded SystemsAlpen-Adria Universität Klagenfurt
2Wilfried Elmenreich
Overview
• Cooperation theory
• The evolutionary approach
• Selection models
• Agent behavior rules
• A tool for evolving SOS
• Application examples
• Challenges
3Wilfried Elmenreich
Cooperation Theory
• Cooperation theory addresses the common tension between – what is good for the individual actor in the short run,
and – what is good for the group in the long run
• Typically employs game theory as the basis for analysis
• Robert Axelrod published Evolution of Cooperation in 1984– Still we are not done with answering this question
today...
4Wilfried Elmenreich
Application Examples
• A soccer team– Discipline, selfishness,
assistance/cooperation…
• Employees/company– „Teamwork“ issue
• Energy consumers and producers in the „Smart Grid“
Source: Wikimedia
5Wilfried Elmenreich
Prisoner‘s Dilemma
• Characterization of 1-move 2-person Prisoner’s Dilemma:
R: the reward for mutual cooperation
P: the punishment for mutual defection T: payoff for unilateral defectionS: the sucker’s payoff
• The Prisoner’s Dilemma is defined by T > R > P > S
• Mutual cooperation should be in favor over coordinated alternation of unilateral cooperation:R > (T+S) / 2
R,R
T,S
S,T
P,P
6Wilfried Elmenreich
Prisoner‘s Dilemma Strategies
• Always defect:– Best for individual actor in the short run– Bad for overall group
• Always cooperate:– Good for group– Rewards antisocial behavior
• A sophisticated strategy– Tit-for-Tat– Forgiveness concept– Retaliation concept
7Wilfried Elmenreich
Public Goods Game
• A social dilemma• Different story, same properties as
Prisoneer‘s Dilemma• Typical much more than 2 players• Altruistic behavior best for group• Best to defect for an individual (Freerider)
– Tragedy of the commons
8Wilfried Elmenreich
Some questions
• What is the most rational strategy?• What are the boundary conditions for a
strategy to be optimal?• Why do real systems converge to equilibria
with non-rational behavior?
• Put your research topic here!
9Wilfried Elmenreich
The Evolutionary Approach
• Research the evolution of rules via Darwinian process
• Model the social game (e.g. Public Goods Game) as the problem
• Evolve the behavioral rules of the agents
10Wilfried Elmenreich
Individual selection based onindividual fitness
• Select the soccer players that score the most goals
• A: altruistic agent• S: selfish agent
A A
AA
A S
S S
SS
S
S S
11Wilfried Elmenreich
Individual selection based ongroup fitness
• Select the best performing agents from the best performing groups
A
S
AS
S
S
A
S
AS
S
S
S
S
S
12Wilfried Elmenreich
Group selection based ongroup fitness
• Select the best performing agents from the best performing groups
A
S
A
S
S
S
A
S
A
A
S
A
A
S
A
13Wilfried Elmenreich
Kin selection
• Hamilton’s rule of inclusive fitness:if r b > c then cooperater relatednessb benefitc cost
• r = probability that two individuals have homologous alleles identical by descent
• J.B.S. Haldane: “I would lay down my life for two brothers or eight cousins.”
14Wilfried Elmenreich
The ongoing discussion: Group vs. Kin selection
• Social insects make a strong point for kin selection– Cooperation without being able to reproduce– But does not cover all evolved cooperations
• Spatial models make a strong point for group selection – Group selection is especially in favor for areas
with natural barriers– Analysis requires now a connection (network)
model
15Wilfried Elmenreich
Searching for the Rules
• Simulation of target system as playground
• Evolvable model of local behavior (e.g., fuzzy rules, ANN)
• Define goal via fitness function (e.g., maximize throughput in a network)
• Run evolutionary algorithm to derive local rules that fulfill the given goal
System modelGoals (fitness function)
Simulation of game
System modelGoals (fitness function)
Simulation of game
Explore solutions
Explore solutions
Evaluate & Iterate
Evaluate & Iterate
Analyze results
Analyze results
16Wilfried Elmenreich
Evolutionary Algorithm
17Wilfried Elmenreich
Agent behavior to be evolved
• Controls the agents of the SOS• Processes inputs (from sensors) and produces
output (to actuators)• Must be evolvable
– Mutation– Recombination
• We cannot easily do this with an algorithm represented in C code…
Agent
Control System„Agent‘s Brain“
18Wilfried Elmenreich
Artificial Neural Networks
• Each neuron sums up theweighted outputs of the other connected neurons
• The output of the neuronis the result of an activation function (e.g. step, sigmoid function) applied to this sum
• Neural networks are distinguished by their connection structure– Feed forward connections (layered)
– Recursive (Ouput neurons feed back to input layer)
– Fully meshed
19Wilfried Elmenreich
Evolving Neural Networks
3.2 -1.2
3.2
3.2
-0.1
-4.2
0.20.0
3.5 -1.2
3.2
3.2
-0.1
-4.2
0.20.0
Mutation
0.0 -1.2
1.2
3.2
-0.1
1.2
1.20.0
3.5 2.2
3.2
3.2
-0.1
-4.2
0.20.5
Recombination
3.2 -1.2
3.2
3.2
-0.1
-4.2
0.20.0
20Wilfried Elmenreich
A Framework for Evolutionary Design
• FREVO (Framework for Evolutionary Design)
• Modular Java tool allowing fast simulation and evolution
• „Frevo“ means also a hot, „boiling“ dance around here
21Wilfried Elmenreich
Framework for Evolutionary Design
• FREVO defines flexible components for – Controller representation
– Problem specification
– Optimizer
22Wilfried Elmenreich
Giving FREVO a Problem
• Basically, we need a simulation of the problem• Interface for input/output connections to the
agents– E.g. for the public goods game:– Your input last round– Your revenue
• Feedback from a simulation run -> fitness value
23Wilfried Elmenreich
Example: Evolving Team Play in Robot Soccer
• Simulated robot soccer teams• Goal evolve behavior for individual players• Team selection strategy
– Not the top scorer but thewinning teams are selected
– This selection forcescooperation within team
24Wilfried Elmenreich
Modeling of individual soccer players
25Wilfried Elmenreich
Agent I/O by example of an ANN
26Wilfried Elmenreich
Composite fitness function
27Wilfried Elmenreich
Results (movie)
http://www.youtube.com/watch?v=cP035M_w82shttp://www.youtube.com/watch?v=cP035M_w82s
28Wilfried Elmenreich
Evolution of cooperative behavior
29Wilfried Elmenreich
Evolution of cooperative behavior
• Public goods game model• 6 players randomly selected from the pool• Neural network model for each player• Player decides how much money to contribute to
a pot• Pot function: money is tripled and evenly
distributed among all• Players have knowledge about their own and
other‘s decisions from previous round
30Wilfried Elmenreich
Freerider behavior evolves
31Wilfried Elmenreich
A synergy effect
• Make the pot function overproportional to the submitted money
• E.g. by a square function: pot = c x2
• This significantly increases the motivation for cooperation
• Does this immediately lead to cooperation?
32Wilfried Elmenreich
Public Goods Game with Synergy Model
• Still nocooperation?
33Wilfried Elmenreich
Cooperation evolves in the long term
• It needs some random fluctuations to set the coopertive strategy loose
• Quickly convergesto (partial) cooperative behavior
34Wilfried Elmenreich
Not the end of the story…
• Spatial model could enable group-selection-like evolution of altruism– But what spatial model?
• 2D von Neumann neighborhood• 2D Moore neighborhood• Hexagon neighborhood• Hypercubes• Network models
• Extending the model with team selection for agents could enable kin selection feature– At least as many questions
35Wilfried Elmenreich
Challenges
• Evolutionary modeling gives new insight to the problem– Answers the question: Why is a particular non-optimal
behavior so prevalent in a population?
• Evolutionary modeling also raises new questions– How should the agent‘s brain be structured?– How should the selection function be modeled?– What is an appropriate spatial model?
36Wilfried Elmenreich
Visit us
• (open source):
www.frevotool.tk• Project MESON (Design Methods for Self-
Organizing Systems):meson.lakeside-labs.com
• Lakeside Labs Clusterwww.lakeside-labs.com