Agent-based Simulation Platform Evaluation in Agent-based Simulation Platform Evaluation in the Context of Human Behavior Modelingthe Context of Human Behavior Modeling

Michal Laclavík, Štefan Dlugolinský, Martin Šeleng, Marcel Kvassay,

Bernhard Schneider, Holger Bracker,

Michał Wrzeszcz, Jacek Kitowski, Ladislav Hluchý

IKT Group - Institute of Informatics SASIKT Group - Institute of Informatics SAS

Dept. of Parallel and Distributed ComputingResearch and Development Areas:

– Large-scale HPCN and Grid applications– Intelligent and Knowledge oriented Technologies

Experience from European projects:– 6 project in FP6: EGEE II, K-Wf Grid, DEGREE

(coordinator), EGEE, int.eu.grid, MEDIGRID– 4 projects in FP7:

Commius, Admire, EGEE III, Secricom– 1 EDA project: EUSAS

Several National Projects (SPVV, VEGA, APVT)IKT Group Focus:

– Multi-Agent Systems– Information Processing– Semantic Web– Knowledge oriented Technologies– Parallel and Distributed

Information ProcessingSolutions:

– AgentOWL: semantic web and FIPA agents– Ontea: Pattern-based Semantic Annotation– ACoMA: KM tool in Email– EMBET: Recommendation System

Director & leader of PDC: Dr. Dipl. Ing. Ladislav Hluchý

URL: http://ikt.ui.sav.sk

2 May 2011 ITMAS 2011 2

3 ITMAS 2011 2 May 2011

EDA R&T JIP FP project (for R&T Joint Investment Programme on Force Protection)

European Urban Simulation

for Asymmetric Scenarios

4 ITMAS 2011 2 May 2011

EUSAS ObjectivesEUSAS Objectives

Training

Modelling

Learning

Analysis

qualitativeevaluation

quantitativeevaluation

RulesOf

Engagement

updates

updates

results

results

results

updatesmodels

interactions

interactions

SeriousGame

DataFarming

ABS Evaluation ApproachABS Evaluation Approach

• Survey Literature– List of available simulation platforms is in Deliverable Appendix

– Existing ABS evaluations were considered

• Evaluation Criteria/Features– 12 features selected – list on the next slide

• Principle: Evaluation through implementation– Exemplary Human Behavior scenario defined

• Civilians getting angry (throwing stones) or afraid (running to safety). • Soldiers arresting Civilians if hit by stone

– Implemented in MASON, NetLogo and VBS2

2 May 2011 ITMAS 2011 5

Survey LiteratureSurvey Literature

• Stupid Agent Model– 16 features

– S.F. Railsback, S.L. Lytinen and S.K. Jackson Agent Based Simulation Platforms: Review and Development Recommendations Simulation 8:9 (2005)

– NetLogo, MASON, Repast, Swarm and Java Swarm

– Later also others, like EcoLab

• Human Behavior Modeling – We have chose 12 features

– Generic, but evaluated on Human Behavior Scenario

2 May 2011 ITMAS 2011 6

12 Features selected as evaluation criteria12 Features selected as evaluation criteria

1. Loading and Representing the Environment and the Scenario

2. Creating and Representing Agents

3. Behavior Implementation

4. Movement Implementation

5. Visualization

6. Parameterization

7. Model check-pointing

8. Analytical Tools

9. Logging

10. Performance

11. Standards

12. Development Environment

2 May 2011 ITMAS 2011 7

Exemplary Human Behavior ScenarioExemplary Human Behavior Scenario

• Soldiers– Catching civilians

if hit twice by stone

– If civilian is caught (arrested), civilian will disappear

– Soldiers are robotic (no emotions)

• Civilians– Driven by 2 emotions (fear and anger)

– When angry, trying to find stone, going to soldier and throws the stone

– When afraid, flying to safety area (yellow)

2 May 2011 ITMAS 2011 8

• Advantages– NetLogo can be invoked and controlled by

another program running on the JVM by Controlling facility API (e.g. app which automates series of model runs, embed NetLogo models in a larger app)

– Simulation state (i.e. world) can be saved to a CSV file and later loaded

– Java API for creating custom extensions to NetLogo (commands, reporters)

– Models can be run without visualization– Ability to load vector GIS data (points, lines,

and polygons - ESRI shapefiles), and raster GIS data (grids) into NetLogo by GIS extension

– Easy to draw graphs, create simulation parameter controllers (sliders, buttons, etc.)

– Many useful tools like BehaviorSpace, System Dynamics Modeler, HubNet, Logging

NETLOGO 4.1NETLOGO 4.1

2 May 2011 ITMAS 2011 9

• Disadvantages

– Slower than Mason, some parts of user code are interpreted at runtime

– The Controlling facility API is considered experimental. It is likely to continue to change and grow.

– Support for creating 3D worlds is still in an experimental state. Only 2D world is fully supported.

NETLOGO 4.1NETLOGO 4.1

2 May 2011 ITMAS 2011 10

• Advantages– Fast, overhead of simulation environment is minimal– Java– Models are completely independent from visualization– Models may be checkpointed and recovered– Agents are not forced to have a physical location,

which is good if we want to create agents representing groups (meso and macro levels)

– Physical environment – any number of 2D or 3D layers– Multiple Displays of simulation– Time series Graphs, variable inspectors– GIS data can be loaded

• Disadvantages– NetLogo has better support for movement and

analysis of distances, objects etc. in physical environment – this impacts development speed, but gives flexibility

MASONMASON

2 May 2011 ITMAS 2011 11

VBS2VBS2

• Integration Challenges

– VBS is thread- and event-based. Our candidate ABS systems (MASON and NetLogo) are step-based. Integration is not straightforward but feasible.

– Changing the action in the middle of its execution may cause a jerkinganimation. For example: while throwing a stone – the agent decides torun to the safety area

– Movement in VBS may be executed a bit differently from what was planned and simulated in ABS: we need to use waypoints to minimize the discrepancy.

2 May 2011 ITMAS 2011 12

Evaluated FeaturesEvaluated Features

• Loading and Representing the Environment and the Scenario

– MASON• 2D, 3D, layered: IntGrid2D, Continuos2D• GIS support: tested

– NetLogo• 2D: two-dimensional grid of “patches”, 3D experimental• easy import from bitmap• GIS support: tested

• Creating and Representing Agents– Soldier, Civilian, Stone

– MASON• Represented by Java class (Steppable interface),

step(SimState state) method• access to environment : SimState state

– NetLogo• Turtles (dynamic), patches, links and the observer

2 May 2011 ITMAS 2011 13

Evaluated FeaturesEvaluated Features

• Behavior Implementation– NetLogo

• turtle variable and the RUN command

– MASON• step(SimState state)

• Movement Implementation– NetLogo

• Direction and step

– MASON• Go to X, Y• Flocking, steering: implemented in demo

• Visualization– MASON: strong separation of Model and Visualization

– NetLogo: possibility to switch off visualization, speed does not change.

2 May 2011 ITMAS 2011 14

Evaluated FeaturesEvaluated Features

• Parameterization– supported

• Model check-pointing– Supported in both

– MASON: platform independent

• Analytical Tools– MASON: improvement over the years

• Property inspectors• Video, snapshot, streaming, charts

– NetLogo:• Property inspectors• snapshot, streaming, charts

• Logging– MASON: log4j can be used

– NetLogo: using log4j integration

2 May 2011 ITMAS 2011 15

Evaluated Features: PerformanceEvaluated Features: Performance

2 May 2011 ITMAS 2011 16

Number of Agents 15 150 1500 15000NetLogo 1 step (ms) 0,48 27,60 18281,95MASON 1 step (ms) 0,10 0,59 21,51 2474,30MASON speed vs. NetLogo 4,8 x 46,8 x 849,9 x

1000 steps (ms)

1000 steps (ms)

10 steps (ms)

Number of Agents: 15 150 15001 run 617 26760 1833232 run 483 27407 1801163 run 474 27804 1753864 run 476 27643 1895735 run 454 28200 1862496 run 470 27759 1801117 run 469 27704 1825268 run 453 27887 1822289 run 461 27230 186770

10 run 476 27629 1819131000 steps (ms): 483 27602 182820

1 step (ms): 0,48 27,60 18281,95

1000 steps (ms)

1000 steps (ms)

1000 steps (ms)

10 steps (ms)

Number of Agents: 15 150 1500 150001 run 98 516 21491 247162 run 109 814 23445 248423 run 99 724 19143 242924 run 96 479 22102 247785 run 90 775 23025 248076 run 91 570 22179 248587 run 87 485 20682 249088 run 99 658 22348 248499 run 130 436 20268 24497

10 run 114 439 20426 248831000 steps (ms): 101 590 21511 2474300

1 step (ms): 0,10 0,59 21,51 2474,30

NetLogo Mason

Evaluated FeaturesEvaluated Features

• Standards– Agent Standards: FIPA – not relevant for simulation agents

– DIS and HLA standards• Relevant but we did not test• VBS2 will be integrated for training

– we plan to use the plug-in functionality in VBS2 and CORBA technology

• need to create a FOM - Federation Object Model• Java based HLA:

– poRTIco

– Java port of CERTI

• Development Environment– Both step based, easy debug, better then thread based MAS

– NetLogo: • NetLogo IDE, debugging using variable inspectors

– MASON: • Any Java IDE, standard Java debbuging• We have used Eclipse

2 May 2011 ITMAS 2011 17

ConclusionConclusion

• Both are almost equal in many features

• NetLogo: better in physical movement support and some analytical tools.

• MASON: much faster, supports strong separation of visualization and behavior models, better support for 3D environment, Java based - easier to integrate with other systems.

Features NetLogo MASONLanguage Logo, Java for simulation control JavaEnviroment 2D, 3D experimental 2D, 3DGIS support Yes YesMovement Heading angle + step just set(x,y)Stearing/Flocking Behaviour Not directly Not directlyVisualization 2D, 2D as 3D 2D, 3Drun with no visualization possible but not strictly separated separated behaviour and visualization modelsParametrization possible possibleModel check-pointing Yes Yes, platform independentAnalytical Tools Charts, Streamning, variable bars, snapshot Charts, Streamning, snapshot, video recordingLogging support using log4j not direct support but log4j can be usedPerformance good for tens of agents good for thouslands of agents

2 May 2011 ITMAS 2011 18