the virtual world framework: implementing a web based client side simulator rob chadwick, katmai...
TRANSCRIPT
The Virtual World Framework: Implementing a Web Based Client
Side Simulator
Rob Chadwick, Katmai Government Services in support of ADL
Outline
IntroductionDesign Principles and RequirementsSystem Architecture
Server componentClient side application
Client system requirementsChallenges and ConsiderationsFuture WorkExamples and ScreenshotsQuestions
2
Introduction
The Virtual World Framework is a project funded by the Office of the Secretary of Defense to build a next generation distributed simulation platform
The intent is to provide an environment for application developers to rapidly create multi-user applications that run in a web browser with no client side software requirements
It must serve as a stable, scalable operating system to enable collaborative training experiences to be rapidly developed and easily
delivered
3
Design Principles and Requirements
Open SourceBoth the client and server side systems are developed with open-source methodologiesThis makes it possible for all manor of users to copy, modify and contribute codeThis also fosters more community interaction and communication
Web BasedThe software does not have any dependencies that must be installed on clients beyond a web browser This should mitigate deployment difficulties
4
Design Principles and Requirements cont.
Client SideAll simulation logic is computed client sideThe server simply connects clients togetherIt may eventually be possible to remove the server completelyAllows for a very lightweight server
Low bandwidthThe VWF uses a principle called ‘replicated computation’ to synchronize clientsThis paradigm timestamps and transmits only inputs to the simulation – allowing each client to independently compute a synchronized stateMessages are queued by the application and executed in order, up to the current time code delivered by the serverClients are not guaranteed to be synchronized in real time, but will remain synchronized relative to the simulation timestamp
5
System Architecture Overview
A VWF application is designed with an MVC ( Model-View-Controller) architectureThe server and client side libraries create a shared model that is automatically synchronized across all clients
Instance
Application
InstanceInstance
Client
Reflector
ViewView
Static Resource
Static Resources
Static Resource
Static Resources
Client
ViewView
ModelModel
6
System Architecture
Composed of two major components – a server and a client applicationServer responsibilities
Serves static resources such as images, 3D models, and soundsOrganizes clients into groups and bridges the communication between clients within a groupDelivers the client side application code, and bootstraps the application Synchronizes late joining clients
Client responsibilitiesExecutes simulation logicInterfaces with userProvides user feedback via graphics, sound, etcBroadcasts user input to the server
7
Server Design
A VWF server can host multiple applicationsEach application might be loaded by several groups of clients at onceNo communication between applications or application instances is allowed
8
nInstance
n
Instance
Client
Virtual World Framework Server
ClientClientClient Client
Instance
ClientClient
Application
Application
Application
Client Application
The client application is a highly modular JavaScript program consisting of a “kernel” and a set of “drivers”These modules work together to execute a simulation in which a hierarchy of nodes modifies its structure and state according to the simulation logicThe kernel orchestrates the interaction of all the nodes and drivers, and forwards messages to the server so that other clients so that they may maintain synchronization.Reusable components can instantiate multiple nodes based on a common prototypeApplication authors may supply custom APIs to the simulation nodes by including additional driversThe VWF supplies a useful set of services to simulation nodes such as scripting, voice and video communications, physics, and audio
Each of these capabilities is implemented within a driver module
9
Client internals
View Drivers
(Your Simulation Code)
Kernel
Model Drivers
Component
Component
Component
Reflector
The VWF system contains ‘drivers’ that provide services to the application‘Model’ drivers participate in the simulation, while ‘view’ drivers produce outputThe ‘kernel’ observes the interaction of these modules and forwards message to the server as necessary to maintain synchronization between clients
10
Client System Requirements
The core VWF kernel has modest system requirementsWebSockets - a protocol for creating bidirectional low latency connections between web servers and clientsECMAScript 5 – the current version of the popular web scripting language (sometimes called JavaScript)
All modern browsers (Chrome, Firefox, Opera, Internet Explorer, Safari) support these featuresOut of the box, the VWF comes with several drivers that have additional requirements
Rendering requires support for WebGL, a 3D graphics interface for web applicationsCommunication services rely on WebRTC and the HTML5 MediaSource APIAudio currently requires HTML5 audio, and may eventually utilize WebAudio
However, only the core components are absolutely necessary for a client to join
11
System Requirements Cont.
PerformanceNo minimum requirementHeavily dependant on the particular logic of a given simulationMay or may not scale with the number of users, depending on the applicationCan be a significant challenge in some situationsSome system configurations will use a software renderer to support WebGL – this can cause issues with complex geometries
BandwidthUsually minimal – only user input data must be transmitted, plus a time pulseCan become prohibitive when using the communication features like voice or video chatImmersive scenes must transmit artwork including 3D models and texturesIt is possible to reduce server bandwidth requirements by hosting assets externally, possibly on a commercial CDNAssets may also be stored locally on the client
12
Challenges in Implementation
Platform supportInitially, this was a significant challengeWhile the base kernel can run almost anywhere, initially support for the 3D rendering feature was sparseSince the beginning of the project, WebGL and other HTML5 features have become much more widely supportedSignificantly, Internet 11 will support WebGL, allowing the VWF to provide immersive 3D visualizations on all major browsersEven Android and IOS tablets now support the requirementsWe expect that further adoption of the web as a platform will mitigate platform support concerns with little direct effort by our staff
13
Challenges in Implementation
PerformanceJavaScript execution speed can still be a bottleneck for some applicationsSpecifically, current JavaScript physics simulators still struggle to scale to large immersive environments on commodity hardwareThere are several interesting projects focused on making JavaScript fasterIt may be possible to compile the VWF client code into a subset of JavaScript called ASM.js, which can be as fast as half native speed on some JS enginesSome of the client side logic may be able to leverage hardware acceleration by utilizing WebCL, a project to bring general purpose GPU acceleration to the web environment.Finally, Google has developed a technology for securely deploying native code on the browser called NativeClient. It may be possible to move some VWF components to this technology
14
Challenges in Implementation
Maintaining synchronization of user scriptsWhen given properly formatted inputs, the system guarantees that each client will have an identical application state at a given time step.However, it is possible for simulation authors to circumvent the synchronization mechanism
By unintentionally storing state information in global variablesBy writing simulation logic that improperly depends on outside inputs
Currently, authors must be careful to follow best practices when creating contentWe’re actively researching long term solutions, such as sandboxing user provided behavior codeWe may also periodically detect synchronization failures by computing and comparing the cryptographic hash of the entire simulation stateIf a mismatch is found, we could re-initialize the clients to a known stateIt may also be possible to detect incorrect user scripts during authoring by analyzing the scripts within an IDE specific to VWF development
15
Future Work
Much remains to be done!The basic platform is complete, and work is focused on providing more supporting features to applications
User management featuresAudio featuresIntegration with outside data sourcesSecure WebSockets and HTTPS content deliveryAutomatic recovery from synchronization failuresSearch and retrieval for reusable components and behaviorsPersistence services for applications
16
Screenshots
Two users interacting on a terrain
17
Screenshots
An environment with several non-player characters
18
Screenshots
A scale representation of the Solar System
19
Questions?
…
20
