24.11.2003 interactive parallel and distributed simulation - dss - jan klír 1 interactive parallel...
TRANSCRIPT
24.11.200324.11.2003 Interactive Parallel and Distributed SimulatInteractive Parallel and Distributed Simulation - DSS - Jan Klírion - DSS - Jan Klír
11
Interactive Parallel & Interactive Parallel & Distributed SimulationDistributed Simulation
24.11.200324.11.2003 Interactive Parallel and Distributed SimulatInteractive Parallel and Distributed Simulation - DSS - Jan Klírion - DSS - Jan Klír
22/39/39
OutlineOutline
Introduction to Parallel and Introduction to Parallel and distributed simulation (basic distributed simulation (basic concepts)concepts)
Methods in Interactive PADSMethods in Interactive PADS• OverviewOverview• ApproachesApproaches
Conclusion, current state, future Conclusion, current state, future workwork
24.11.200324.11.2003 Interactive Parallel and Distributed SimulatInteractive Parallel and Distributed Simulation - DSS - Jan Klírion - DSS - Jan Klír
33/39/39
Area DefinitionArea Definition
Time-steppedTime-stepped: : time advances by time advances by fixed time fixed time incrementsincrements
Event drivenEvent driven: : time advances time advances occur with irregular occur with irregular incrementsincrements
computersimulation
discretemodels
continuousmodels
eventdriven
time-stepped
24.11.200324.11.2003 Interactive Parallel and Distributed SimulatInteractive Parallel and Distributed Simulation - DSS - Jan Klírion - DSS - Jan Klír
44/39/39
Distributed SimulationDistributed Simulation
As fast as possibleAs fast as possible Analytical Analytical
simulationssimulations High Performance High Performance
Computing Computing CommunityCommunity
Conservative and Conservative and Optimistic (Time Optimistic (Time Warp) Warp) synchronizationsynchronization
RealtimeRealtime Virtual Virtual
environments, environments, hardware-in-the-hardware-in-the-looploop
Defense M&S Defense M&S communitycommunity
SIMNET -> DIS -> SIMNET -> DIS -> HLAHLA
24.11.200324.11.2003 Interactive Parallel and Distributed SimulatInteractive Parallel and Distributed Simulation - DSS - Jan Klírion - DSS - Jan Klír
55/39/39
Conservative SynchronizationConservative Synchronization
LP 4
LP 3
LP 2
LP 1
Simulation TimeT
events ready to be processed
T+L
•L – L – LookaheadLookahead – Any message sent by a LP must have – Any message sent by a LP must have timestamp at least T + Ltimestamp at least T + L•Necessary to allow concurrent processing of events with different time stamps•Relies on model props, complicates the model developmentRelies on model props, complicates the model development
events not ready to be processed
24.11.200324.11.2003 Interactive Parallel and Distributed SimulatInteractive Parallel and Distributed Simulation - DSS - Jan Klírion - DSS - Jan Klír
66/39/39
Optimistic SynchronizationOptimistic Synchronization
LP 4
LP 3
LP 2
LP 1
Simulation TimeT
processed events
unprocessed events
rollback
•LP processes events without checking if an event in its past LP processes events without checking if an event in its past could later arrivecould later arrive•If an event arrives in the LPs past (If an event arrives in the LPs past (straggler messagestraggler message), the ), the computation is rolled backcomputation is rolled back
24.11.200324.11.2003 Interactive Parallel and Distributed SimulatInteractive Parallel and Distributed Simulation - DSS - Jan Klírion - DSS - Jan Klír
77/39/39
Optimistic SynchronizationOptimistic Synchronizationlocal control mechanismlocal control mechanism
Input Queue(event list)
straggler message arrives in the past, causing rollback
12 21 35
processed event
unprocessed event
saved state
anti-messageState Queue
1212Output Queue(anti-messages)
•LP processes events in time order like sequential simulatorLP processes events in time order like sequential simulator•The state before processed events must be saved, the The state before processed events must be saved, the messages sent to other LPs must be also savedmessages sent to other LPs must be also saved•When a message arrives in the past (straggler message), the When a message arrives in the past (straggler message), the rollback is initiatedrollback is initiated
24.11.200324.11.2003 Interactive Parallel and Distributed SimulatInteractive Parallel and Distributed Simulation - DSS - Jan Klírion - DSS - Jan Klír
88/39/39
Optimistic SynchronizationOptimistic Synchronizationconcepts and notionsconcepts and notions
State savingState saving – various algorithms – various algorithms (Copy SS, Periodic SS, Incremental (Copy SS, Periodic SS, Incremental SS, …)SS, …)
Anti messagesAnti messages• In case of rollback are sent to In case of rollback are sent to
destination LPsdestination LPs• If LP gets anti message, the If LP gets anti message, the
corresponding waiting event is removed corresponding waiting event is removed from the queue (from the queue (annihilationannihilation))
24.11.200324.11.2003 Interactive Parallel and Distributed SimulatInteractive Parallel and Distributed Simulation - DSS - Jan Klírion - DSS - Jan Klír
99/39/39
Optimistic SynchronizationOptimistic Synchronizationconcepts and notionsconcepts and notions
Global virtual timeGlobal virtual time• Minimum timestamp of any unprocessed Minimum timestamp of any unprocessed
message or anti-message in the whole systemmessage or anti-message in the whole system• Used to reclaim memory used by SS and Used to reclaim memory used by SS and
output queues (output queues (fossil collectionfossil collection))• Also serves to commit blocking I/O operationsAlso serves to commit blocking I/O operations• Synchronous X Asynchronous GVT computationSynchronous X Asynchronous GVT computation• Wide spectrum of GVT algorithms exist today, Wide spectrum of GVT algorithms exist today,
the selection depends on application. The the selection depends on application. The algorithms are non-trivial in distributed algorithms are non-trivial in distributed environments.environments.
24.11.200324.11.2003 Interactive Parallel and Distributed SimulatInteractive Parallel and Distributed Simulation - DSS - Jan Klírion - DSS - Jan Klír
1010/39/39
PADS ConclusionPADS Conclusion
Conservative or Optimistic approachConservative or Optimistic approach Optimistic much more used than Optimistic much more used than
conservativeconservative
Next part: Interactive methods based Next part: Interactive methods based on PADSon PADS
24.11.200324.11.2003 Interactive Parallel and Distributed SimulatInteractive Parallel and Distributed Simulation - DSS - Jan Klírion - DSS - Jan Klír
1111
Interactive simulationInteractive simulation
Methods in Interactive PADSMethods in Interactive PADS
24.11.200324.11.2003 Interactive Parallel and Distributed SimulatInteractive Parallel and Distributed Simulation - DSS - Jan Klírion - DSS - Jan Klír
1212/39/39
Interactive simulation - outlineInteractive simulation - outline
Introduction – applications & Introduction – applications & requirementsrequirements
Description of Interactive PADS Description of Interactive PADS approachesapproaches• Multiple path explorationMultiple path exploration• Input / outputInput / output• Variable resolution modelingVariable resolution modeling
Summary, open issues, current stateSummary, open issues, current state
24.11.200324.11.2003 Interactive Parallel and Distributed SimulatInteractive Parallel and Distributed Simulation - DSS - Jan Klírion - DSS - Jan Klír
1313/39/39
Applications of Interactive Applications of Interactive SimulationSimulation
Decision support - (Air) traffic control, Decision support - (Air) traffic control, planning, …planning, …
Interactive system analysis – Interactive system analysis – Networks, transportation systems, Networks, transportation systems, ecological systems, logistics, ecological systems, logistics, battlefield, …battlefield, …
Virtual environments – Training Virtual environments – Training simulators, medicine (virtual simulators, medicine (virtual surgery), … surgery), …
24.11.200324.11.2003 Interactive Parallel and Distributed SimulatInteractive Parallel and Distributed Simulation - DSS - Jan Klírion - DSS - Jan Klír
1414/39/39
RequirementsRequirementswhat should an interactive simulator supportwhat should an interactive simulator support
Exploration of different simulation Exploration of different simulation futuresfutures• Rewind to some point and continue with Rewind to some point and continue with
different inputdifferent input• Execute multiple simulations Execute multiple simulations
concurrentlyconcurrently Fast input and outputFast input and output Variable resolution models Variable resolution models
• Interactive detail level changingInteractive detail level changing
24.11.200324.11.2003 Interactive Parallel and Distributed SimulatInteractive Parallel and Distributed Simulation - DSS - Jan Klírion - DSS - Jan Klír
1515/39/39
Multiple Path ExplorationMultiple Path Exploration
Uses – explore different simulation Uses – explore different simulation paths based on different input paths based on different input parametersparameters
ApproachesApproaches• Multiple sequential simulations with Multiple sequential simulations with
different parametersdifferent parameters• Interactive simulation steering with Interactive simulation steering with
rollback & reexecutionrollback & reexecution• Cloning (Virtual processes)Cloning (Virtual processes)
24.11.200324.11.2003 Interactive Parallel and Distributed SimulatInteractive Parallel and Distributed Simulation - DSS - Jan Klírion - DSS - Jan Klír
1616/39/39
Rollback & ReexecutionRollback & Reexecution
Input events called ‘steering’ events – Input events called ‘steering’ events – modify the simulation path according the modify the simulation path according the inputinput
Special ‘reexecution’ events allow to Special ‘reexecution’ events allow to rewind to the pastrewind to the past• Scheduled to the pastScheduled to the past• The simulation can take a new path by The simulation can take a new path by
inputting new data via steering eventsinputting new data via steering events• Suitable for Time Warp simulators that utilize Suitable for Time Warp simulators that utilize
the rollback as a part of the synchronizationthe rollback as a part of the synchronization
24.11.200324.11.2003 Interactive Parallel and Distributed SimulatInteractive Parallel and Distributed Simulation - DSS - Jan Klírion - DSS - Jan Klír
1717/39/39
Rollback & ReexecutionRollback & Reexecution
ProblemsProblems• There must be a saved state prior the There must be a saved state prior the
time of the reexecution eventtime of the reexecution event• The available rollback length is limited - The available rollback length is limited -
due to fossil collection in Time Warp due to fossil collection in Time Warp simulators the old saved states are simulators the old saved states are reclaimed as the GVT passesreclaimed as the GVT passes
• Some state saving schemes were Some state saving schemes were developed but the rollback is still limiteddeveloped but the rollback is still limited
24.11.200324.11.2003 Interactive Parallel and Distributed SimulatInteractive Parallel and Distributed Simulation - DSS - Jan Klírion - DSS - Jan Klír
1818/39/39
Cloning (Virtual Processes)Cloning (Virtual Processes)
Multiple simulation futures are computed Multiple simulation futures are computed in parallelin parallel
The futures share the computation until The futures share the computation until they branch with different parametersthey branch with different parameters
Uses the Virtual processes paradigmUses the Virtual processes paradigm• All the computed simulations contain virtual All the computed simulations contain virtual
processesprocesses• All corresponding virtual processes are mapped All corresponding virtual processes are mapped
to a single physical process, until the branchto a single physical process, until the branch
24.11.200324.11.2003 Interactive Parallel and Distributed SimulatInteractive Parallel and Distributed Simulation - DSS - Jan Klírion - DSS - Jan Klír
1919/39/39
Virtual ProcessesVirtual Processes
2 virtual simulations, cloned on process A2 virtual simulations, cloned on process A
A2 B2C22
VA1 VB1 VC1
VA2 VB2 VC2
PA2PB1 PC1
PA1
A1 B1
C11
Virtual simulation planesVirtual simulation planes Mapping to phys. processes
24.11.200324.11.2003 Interactive Parallel and Distributed SimulatInteractive Parallel and Distributed Simulation - DSS - Jan Klírion - DSS - Jan Klír
2020/39/39
Virtual ProcessesVirtual Processes
CommunicationCommunication• B -> C: message to PB -> C: message to PCC and thus to V and thus to VC1C1
and Vand VC2C2
• B -> A: message to PB -> A: message to PA1A1 (V (VA1A1) and copied ) and copied to Pto PA2A2 (V (VA2A2))
• A -> B (A1 to B1): PA -> B (A1 to B1): PB2B2 created and created and message sent to Pmessage sent to PB1B1 (V (VB1B1))
24.11.200324.11.2003 Interactive Parallel and Distributed SimulatInteractive Parallel and Distributed Simulation - DSS - Jan Klírion - DSS - Jan Klír
2121/39/39
CloningCloning
3 phases: Both simulations identical; 3 phases: Both simulations identical; Incremental cloning; Both Incremental cloning; Both simulations fully clonedsimulations fully cloned
Cloning rate depends on the amount Cloning rate depends on the amount of communication of communication
24.11.200324.11.2003 Interactive Parallel and Distributed SimulatInteractive Parallel and Distributed Simulation - DSS - Jan Klírion - DSS - Jan Klír
2222/39/39
Multiple Path ExplorationMultiple Path Explorationsummarysummary
Reexecution more interactive, cloning Reexecution more interactive, cloning faster (offers more parallelism)faster (offers more parallelism)
Reexecution - limited rollback length; Reexecution - limited rollback length; cloning – all paths must be known before cloning – all paths must be known before the branchthe branch
Open issuesOpen issues• Greater rollback length, merging convergent Greater rollback length, merging convergent
sim. paths, create new branch point in the past sim. paths, create new branch point in the past and keep the current path, …and keep the current path, …
24.11.200324.11.2003 Interactive Parallel and Distributed SimulatInteractive Parallel and Distributed Simulation - DSS - Jan Klírion - DSS - Jan Klír
2323/39/39
Input / OutputInput / Output
Purpose: To provide accurate and Purpose: To provide accurate and fast I/O to distributed simulatorsfast I/O to distributed simulators
Several classes of I/O events:Several classes of I/O events:• Blocking X non-blockingBlocking X non-blocking• Various types of access to the Various types of access to the
simulation statesimulation state Speed-up: optimistic outputSpeed-up: optimistic output
24.11.200324.11.2003 Interactive Parallel and Distributed SimulatInteractive Parallel and Distributed Simulation - DSS - Jan Klírion - DSS - Jan Klír
2424/39/39
I/O EventsI/O Events
3 classes of output events:3 classes of output events:• Sampling: provide a subset of state information Sampling: provide a subset of state information
in regular intervalsin regular intervals• Notification: occur in predefined transient Notification: occur in predefined transient
conditioncondition• Queries: require access to arbitrary state in Queries: require access to arbitrary state in
previously unknown timepreviously unknown time Input:Input:
• Events injected by external entitiesEvents injected by external entities• Blocking requests for input scheduled by the Blocking requests for input scheduled by the
simulatorsimulator
24.11.200324.11.2003 Interactive Parallel and Distributed SimulatInteractive Parallel and Distributed Simulation - DSS - Jan Klírion - DSS - Jan Klír
2525/39/39
Optimistic OutputOptimistic Output
Output is often visually rich and Output is often visually rich and computation intensivecomputation intensive
Speedup:Speedup:• Output is non-blocking eventOutput is non-blocking event• It’s optimistically precomputed and It’s optimistically precomputed and
thrown away in case of rollbackthrown away in case of rollback• Good for faster-than-realtime simulators Good for faster-than-realtime simulators
and computationally intensive outputand computationally intensive output
24.11.200324.11.2003 Interactive Parallel and Distributed SimulatInteractive Parallel and Distributed Simulation - DSS - Jan Klírion - DSS - Jan Klír
2626/39/39
Input / OutputInput / Outputconclusionconclusion
Open issuesOpen issues• Queries in distributed simulation, over a Queries in distributed simulation, over a
wide time rangewide time range• Optimistic output:Optimistic output:
Limiting the optimism to save resourcesLimiting the optimism to save resources Relationship between optimistic output and Relationship between optimistic output and
the probability of inputthe probability of input
24.11.200324.11.2003 Interactive Parallel and Distributed SimulatInteractive Parallel and Distributed Simulation - DSS - Jan Klírion - DSS - Jan Klír
2727/39/39
Variable Resolution ModelingVariable Resolution Modeling
Why use VRM?Why use VRM?• Speed-up – execute the focused portion Speed-up – execute the focused portion
at full detail and the other at a lower d.at full detail and the other at a lower d.• Zooming – overview the whole and focus Zooming – overview the whole and focus
on details to better comprehend the on details to better comprehend the simulated phenomenonsimulated phenomenon
• Whole-greater-than-sum-of-parts Whole-greater-than-sum-of-parts relationshiprelationship
• Combine different legacy models with Combine different legacy models with diff. detail -> heterogeneous sim. diff. detail -> heterogeneous sim.
24.11.200324.11.2003 Interactive Parallel and Distributed SimulatInteractive Parallel and Distributed Simulation - DSS - Jan Klírion - DSS - Jan Klír
2828/39/39
Variable Resolution ModelingVariable Resolution Modelingapproachesapproaches
Selective viewingSelective viewing• Simulate only the most detailed model Simulate only the most detailed model • Calculate the other views from the most Calculate the other views from the most
detailed modeldetailed model Aggregation/DisaggregationAggregation/Disaggregation
• One level is executed at one timeOne level is executed at one time• When needed, low resolution entity is When needed, low resolution entity is
replaced with a set of high res. entities replaced with a set of high res. entities (disaggregation) and vice versa(disaggregation) and vice versa
24.11.200324.11.2003 Interactive Parallel and Distributed SimulatInteractive Parallel and Distributed Simulation - DSS - Jan Klírion - DSS - Jan Klír
2929/39/39
Variable Resolution ModelingVariable Resolution Modelingapproachesapproaches
Concurrent representationConcurrent representation• All the entities are in the model All the entities are in the model
concurrentlyconcurrently• Only the selected detail level is Only the selected detail level is
executedexecuted• Other detail levels maintain consistency Other detail levels maintain consistency
by transforming the results from the by transforming the results from the running entitiesrunning entities
Existing methods use mix of these Existing methods use mix of these approachesapproaches
24.11.200324.11.2003 Interactive Parallel and Distributed SimulatInteractive Parallel and Distributed Simulation - DSS - Jan Klírion - DSS - Jan Klír
3030/39/39
Multi Representation EntitiesMulti Representation Entities
Single MRE has Single MRE has multiple detail levelsmultiple detail levels
Communication with Communication with other entities at other entities at appropriate levelappropriate level
Consistency Consistency maintained by C.E. maintained by C.E. which has a set of which has a set of mapping functions to mapping functions to convert the state convert the state valuesvalues
MRE E1
A
B1 B1 B1
Consistencyenforcer
E2
E3
24.11.200324.11.2003 Interactive Parallel and Distributed SimulatInteractive Parallel and Distributed Simulation - DSS - Jan Klírion - DSS - Jan Klír
3131/39/39
Hooking TransactionsHooking Transactions
Model designed at multiple detail Model designed at multiple detail levelslevels
At the run time, the channel leading At the run time, the channel leading to the LRE can be redirected to a to the LRE can be redirected to a HRE to increase the detail (zoom in)HRE to increase the detail (zoom in)
The disaggregated LRE can continue The disaggregated LRE can continue or is stripped completelyor is stripped completely
Simulation language primitives exist Simulation language primitives exist to be used by modelersto be used by modelers
24.11.200324.11.2003 Interactive Parallel and Distributed SimulatInteractive Parallel and Distributed Simulation - DSS - Jan Klírion - DSS - Jan Klír
3232/39/39
Hooking TransactionsHooking Transactions
LRE 1 LRE 2 LRE 3
HRE 1
24.11.200324.11.2003 Interactive Parallel and Distributed SimulatInteractive Parallel and Distributed Simulation - DSS - Jan Klírion - DSS - Jan Klír
3333/39/39
Hooking TransactionsHooking Transactions
Consistency: Attribute ghostingConsistency: Attribute ghosting• Attributes of the striped entity are Attributes of the striped entity are
calculated from the newly hooked entitycalculated from the newly hooked entity Mapping consistencyMapping consistency
• The state of the HRE on disaggregation The state of the HRE on disaggregation must be properly initialized from the must be properly initialized from the LRE’s state (and vice versa) LRE’s state (and vice versa)
24.11.200324.11.2003 Interactive Parallel and Distributed SimulatInteractive Parallel and Distributed Simulation - DSS - Jan Klírion - DSS - Jan Klír
3434/39/39
Aggregation / DisaggregationAggregation / Disaggregation
Used often in the HLA areaUsed often in the HLA area Low resolution entities (aircraft patrols) Low resolution entities (aircraft patrols)
automatically disaggregate to a set of high automatically disaggregate to a set of high resolution entities (single aircrafts) when resolution entities (single aircrafts) when approaching other HREs (ground radars)approaching other HREs (ground radars)
HLA is enriched with a set of functions that HLA is enriched with a set of functions that enable the registration of HREs and LREs, enable the registration of HREs and LREs, together with transformation functions and together with transformation functions and disaggregation conditionsdisaggregation conditions
24.11.200324.11.2003 Interactive Parallel and Distributed SimulatInteractive Parallel and Distributed Simulation - DSS - Jan Klírion - DSS - Jan Klír
3535/39/39
Aggregation / DisaggregationAggregation / Disaggregation
LRE 1 LRE 3LRE 2HRE2
HRE1
HRE3
24.11.200324.11.2003 Interactive Parallel and Distributed SimulatInteractive Parallel and Distributed Simulation - DSS - Jan Klírion - DSS - Jan Klír
3636/39/39
Other ApproachesOther Approaches
Optimization based MRMOptimization based MRM• Models at different resolution level Models at different resolution level
provide a set of inputsprovide a set of inputs• Mathematical function describing Mathematical function describing
differences between the outputs is differences between the outputs is defineddefined
• Using mathematical analysis the inputs Using mathematical analysis the inputs are calculated so the differences are calculated so the differences between the outputs are minimizedbetween the outputs are minimized
24.11.200324.11.2003 Interactive Parallel and Distributed SimulatInteractive Parallel and Distributed Simulation - DSS - Jan Klírion - DSS - Jan Klír
3737/39/39
Variable Resolution ModelingVariable Resolution Modelingsummarysummary
VRM – rich area with variety of possible VRM – rich area with variety of possible applicationsapplications
Hooking Hooking • Highly interactiveHighly interactive• Currently only for sequential simulationCurrently only for sequential simulation
MREMRE• Solid designSolid design• Not widely used (high complexity and demands on Not widely used (high complexity and demands on
modelers)modelers) Aggregation/DisaggregationAggregation/Disaggregation
• Oldest approach, simpleOldest approach, simple• Good for real-time simulation area (HLA) Good for real-time simulation area (HLA)
24.11.200324.11.2003 Interactive Parallel and Distributed SimulatInteractive Parallel and Distributed Simulation - DSS - Jan Klírion - DSS - Jan Klír
3838/39/39
Current StateCurrent Stateand future workand future work
Prof. Prof. ŠafaříkŠafařík - 2 diploma theses to - 2 diploma theses to incorporate optimistic and incorporate optimistic and conservative synchronization to J-Sim conservative synchronization to J-Sim ((C-Sim) C-Sim)
Base on this and incorporate the Base on this and incorporate the interactivity as shown in this interactivity as shown in this presentation, trying to overcome the presentation, trying to overcome the shortcomings of the presented shortcomings of the presented approachesapproaches
24.11.200324.11.2003 Interactive Parallel and Distributed SimulatInteractive Parallel and Distributed Simulation - DSS - Jan Klírion - DSS - Jan Klír
3939
The EndThe EndQuestions ???Questions ???
Some pictures were reproduced from the Parallel and Distributed Simulation Some pictures were reproduced from the Parallel and Distributed Simulation course by M. Hybinette, PADS presentations by R. Fujimoto, and from various course by M. Hybinette, PADS presentations by R. Fujimoto, and from various papers cited in the papers cited in the Methods in Interactive Parallel and Distributed Simulation Methods in Interactive Parallel and Distributed Simulation
paper paper