reflection for tools development
DESCRIPTION
Reflection for Tools Development. Jeremy Walker. Senior Tools Programmer Ubisoft Vancouver. Table of Contents. Understanding the Costs of Tools Development The Ultimate Technique for Data Driven Solutions A Reflection-Powered Infrastructure for Tools. 1. The Problem. 2. Reflection. - PowerPoint PPT PresentationTRANSCRIPT
Reflection for Tools Development
Senior Tools ProgrammerUbisoft Vancouver
Jeremy Walker
• Understanding the Costs of Tools Development
• The Ultimate Technique for Data Driven Solutions
• A Reflection-Powered Infrastructure for Tools
Table of Contents
1. The Problem
2. Reflection
3. Content Framework
1. The ProblemUnderstanding the Costs of Tools Development
Hard Coded vs. Data Driven Systems
Animation
Hard Coded Animation System
Animation
Animation Tools
Animation Build System
Animation Engine
Animation Serialization
Hard Coded System Data Driven System
What's the Cost of Great Workflows?
Scenes
Scene Editing Tools
Scene Build System
Scene Rendering Engine
Scene Serialization
Animation
Animation Tools
Animation Build System
Animation Engine
Animation Serialization
Sounds
Sound Editing Tools
Sound Build System
Sound Playback Engine
Sound Serialization
SHARED SHAREDSHAREDSHARED SHARED
SHAREDSHARED
SHAREDSHAREDSHARED
SHARED
SHAREDSHAREDSHAREDSHARED
SHAREDSHAREDSHARED
HUD
Animation Sounds
Cut Scenes
Input Front End Cameras Particles Characters
Textures AI Levels Models Shaders Materials Props
Skeletons Music Sequences State Flows Curves Videos Scripts
Scenes
Great Workflows for All Types of Content
SHARED SHAREDSHAREDSHARED SHARED
SHAREDSHARED
SHAREDSHAREDSHARED
SHARED
SHAREDSHAREDSHAREDSHARED
SHAREDSHAREDSHARED
HUD
Animation Sounds
Cut Scenes
Input Front End Cameras Particles Characters
Textures AI Levels Models Shaders Materials Props
Skeletons Music Sequences State Flows Curves Videos Scripts
Scenes
Hard Coded
Hard Coded
Hard Coded
Hard Coded
Hard Coded
Hard Coded
Hard Coded
Hard Coded
Hard Coded
Hard Coded
Hard Coded
The Keep-It-Simple Approach
Hard Coded
HUD
Animation Sounds
Cut Scenes
Input Front End Cameras Particles Characters
Textures AI Levels Models Shaders Materials Props
Skeletons Music Sequences State Flows Curves Videos Scripts
Scenes
The Monolithic Engine ApproachMonolithic Engine
Animation, Sounds, Scenes, Input, Front End, Cameras, Particles, Characters, Textures, AI, Levels, Models, Shaders, Materials, HUD, Props, Skeletons, Music, Sequences, State
Flows, Curves, Videos, Scripts, Cut Scenes
Animation, Sounds, Scenes, Input, Front End, Cameras, Particles, Characters, Textures, AI, Levels, Models, Shaders, Materials, HUD, Props, Skeletons, Music, Sequences, State
Flows, Curves, Videos, Scripts, Cut Scenes
Animation, Sounds, Scenes, Input, Front End, Cameras, Particles, Characters, Textures, AI, Levels, Models, Shaders, Materials, HUD, Props, Skeletons, Music, Sequences, State
Flows, Curves, Videos, Scripts, Cut Scenes
Animation, Sounds, Scenes, Input, Front End, Cameras, Particles, Characters, Textures, AI, Levels, Models, Shaders, Materials, HUD, Props, Skeletons, Music, Sequences, State
Flows, Curves, Videos, Scripts, Cut Scenes
The Problem
LowDevelopment
Cost
Stable Tools
ReusableSystems
Great Workflows
AdaptableTo Change
Game andGenre-Specific
Needs
?
For all types of content:• Minimize the cost of developing great workflows.• Design reusable systems while accommodating game-
specific needs.• Develop stable tools that adapt to constant change
during production.
The Solution
The Package Release Cycle
Source Code
Automated Build Process OK?
Release Candidate
yes
no
Testing Process OK?
yes
no
Release
Artist Gets Package Release
Programmer Modifies Code
Artist Requests New
Features
The Monolithic Package Release Problem
Monolithic Package Source
Code
Monolithic Package Release
Package Release
Package Release
Package Release
Package Release
Package Release
The Benefit of Decoupled PackagesPackage
Source Code
Package Source Code
Package Source Code
Package Source Code
Package Source Code
Animation Tool Release
Sound Tool Release
Game Release
The Lockstep Release Problem
Animation Tool Source Code
Sound Tool Source Code
Game Source Code
Compile TimeDependency
Compile TimeDependency
Lockstep Release Cycle
Run TimeDependency
Run TimeDependency
Decoupled Systems
HUD
Animation Sounds
Cameras Particles Characters
Materials Props
Scenes
Decoupled Systems
HUD
Animation Sounds
Cameras Particles Characters
Materials Props
Scenes
Reflection
Reducing Cost and Complexity
Hardcoded Engine
Hard Coded
Low CostSimple
Hard to ReusePoor Workflow
Monolithic Engine
Animation, Sounds, Scenes, Input, Front End, Cameras, Particles, Characters, Textures, AI, Levels, Models, Shaders, Materials, HUD, Props, Skeletons, Music,
Sequences, State Flows, Curves, Videos, Scripts, Cut Scenes
Animation, Sounds, Scenes, Input, Front End, Cameras, Particles, Characters, Textures, AI, Levels, Models, Shaders, Materials, HUD, Props, Skeletons, Music,
Sequences, State Flows, Curves, Videos, Scripts, Cut Scenes
Animation, Sounds, Scenes, Input, Front End, Cameras, Particles, Characters, Textures, AI, Levels, Models, Shaders, Materials, HUD, Props, Skeletons, Music,
Sequences, State Flows, Curves, Videos, Scripts, Cut Scenes
Animation, Sounds, Scenes, Input, Front End, Cameras, Particles, Characters, Textures, AI, Levels, Models, Shaders, Materials, HUD, Props, Skeletons, Music,
Sequences, State Flows, Curves, Videos, Scripts, Cut Scenes
Medium CostComplex
Reuse All or NothingGreat Workflow
High CostVery ComplexVery Reusable
Great Workflow
Low CostSimple
Very ReusableGreat Workflow
2. ReflectionThe Ultimate Technique
for Data Driven Solutions
From Wikipedia: “…Reflection is the process by which a computer program can observe and modify its own structure and behaviour.”
What is Reflection?
Reflection is Used Everywhere
Case Study: C++ Compilation
C++Compiler
Function Bcalls
Function A
ExposeDiscover
Validate Implement
Reflection
Function Declaration
Binding to Function
Definition
Function AC++Compiler
Reflection for Language Interoperability
Visual BasicCompilerC# Code
ExposeDiscover
Validate Implement
Reflection
Type Definitions
Type Binding
Visual Basic Code
C#Compiler
Reflection for Scripting in Games
C++ AdapterLua Script
ExposeDiscover
Validate Implement
Type Definitions
Type Binding
C++ ClassesScript Adapter
Reflection
Reflection for Serialization in Games
C++ Adapter
Serialized Objects
ExposeDiscover
Validate Implement
Type Definitions
Type Binding
C++ ClassesSerializer Reflection
Common Language Specification
Type Definitions Type BindingReflection
• Type size and alignment• Native types• Classes• Enums
• Allocate objects• Binary representation• Invoke constructor, get/invoke members• Get/set enum value
const Type* type = TypeOf<SimpleVehicle>();const char* name = type->Name();size_t size = type->Size();printf(“Type: %s\n Size: %d\n”, name, size);
Instance vehicle = type->CreateInstance();vehicle.As<SimpleVehicle&>().m_MaxSpeedKPH = 193.12f;for (int i=0; i<type->NumMembers(); i++){
const Member* member = vehicle.GetMember(i);if (member->IsField()){
const Field* field = (Field*)member;if (field->GetType() == TypeOf<float>()){
Instance value = vehicle.GetField(field);printf(“Field: %s\n Offset: %d\n Value: %.2f\n”,
field->Name(), field->Offset(), value.As<float>());}
}}
Example: Print all fields of type “float”
Reflection in Action
Type: SimpleVehicle Size: 4Field: MaxSpeedKPH Offset 0 Value: 193.12
Output
• Macros• Code Parser• Type Definition Language
Popular Approaches to C++ Reflection
Reflection with Macros
class SimpleVehicle : public Entity{public:
DECLARE_TYPE(); float m_MaxSpeedKPH; void Reset(bool useDefaults);
float GetMaxSpeedMPH() const;void SetMaxSpeedMPH(float maxSpeedMPH);
};
//In a separate .CPP file:DEFINE_TYPE(SimpleVehicle)
BASE_CLASS(Entity)FIELD(“MaxSpeedKPH”, m_MaxSpeedKPH)METHOD(Reset)PROPERTY(GetMaxSpeedMPH, SetMaxSpeedMPH)
DEFINE_TYPE_END()
Macros Example Pros• No external toolsCons• Awkward to implement• Hard to debug• Run-time discovery
Reflection with Code Parser
/// [Class]class SimpleVehicle : public Entity{public:
/// [Field(“MaxSpeedKPH”)]float m_MaxSpeedKPH;/// [Method]void Reset(bool useDefaults);/// [Property]float GetMaxSpeedMPH() const;/// [Property]void SetMaxSpeedMPH(float maxSpeedMPH);
};
Code Parser Example Pros• Easier to implement• Compile-time discoveryCons• Slow pre-build step
Reflection with Type Definition Language
class SimpleVehicle : Entity{
float MaxSpeedKPH;void Reset(bool useDefaults);float MaxSpeedMPH { get; set; }
};
Type Definition Language Example Pros• Easiest to implement• No slow pre-buildCons• Can’t reflect existing
classes
/// [Class]class SimpleVehicle : public Entity{public:
/// [Field(“MaxSpeedKPH”)]float m_MaxSpeedKPH;/// [Method]void Reset(bool useDefaults);/// [Property]float GetMaxSpeedMPH() const;/// [Property]void SetMaxSpeedMPH(float maxSpeedMPH);
};
C++ Class<type name=“SimpleVehicle”> <field name=“MaxSpeedKPH” type=“float”/> <method name=“Reset” returntype=“void”> <parameter name=“useDefaults” type=“bool”/> </method> <property name=“MaxSpeedMPH” type=“float” hasget=“true” hasset=“true”/></type>
Types.xml
Exporting Type Definitions for Game
C++ Reflected Types
(game-side)Code Parser
Types.xml
Generated Binder Code
Reflection System(game-side)
<type name=“SimpleVehicle”> <field name=“MaxSpeedKPH” type=“float”/> <method name=“Reset” returntype=“void”> <parameter name=“useDefaults” type=“bool”/> </method> <property name=“MaxSpeedMPH” type=“float” hasget=“true” hasset=“true”/></type>
Types.xml
Exporting Type Definitions for Tools
C++ Reflected Types
(game-side)Code Parser Types.xml Reflection System
(tool-side)C# Proxy
Generator
[ProxyType(“SimpleVehicle”, 0x81c37132)]public partial class SimpleVehicle : Entity{ public float MaxSpeedKPH { get { return this.Instance.GetField(“MaxSpeedKPH”).Get<float>(); } set { this.Instance.GetField(“MaxSpeedKPH”).Set<float>(value); } } } public float MaxSpeedMPH { ... } public void Reset(bool useDefaults) { ... }}
C# Proxy Class
Primary Uses for Reflection in Tools
Types.xml
Reflection SystemSerialization
Generated UI(PropertyGrid, TreeView, etc.)
Client-ServerRemoting
Client-Server Remoting
RemotingClient
Generated C# Proxy Classes
ExposeDiscover
Validate Implement
Reflection
Type Definitions
Type Binding
C# Proxy Generator C++ Adapter
ExposeDiscover
Validate Implement
Reflection
Type Definitions
Type Binding
C++ Game Classes
RemotingServer
Tools (Client) Game (Server)
Type definitions out of sync• Detect type checksum mismatch• Detect problems early• Auto-synchronization of type information• Auto-migration of data
Problems and Workarounds
Tools tightly coupled to the game• Avoid overuse of generated proxy classes• Use generated UI where possible• Use polymorphic proxy classes
Problems and Workarounds
Tools Code Base Class
Derived Class A
Derived Class B
Derived Class C
has proxy
has no proxy
Excessive memory usage• Strip type information based on usage • Auto-detect unused reflected types
Problems and Workarounds
• Marshalling events for multi-processor architectures• Client-Server Remoting for Online• Serialization of saved game data
Other Uses for Reflection
3. Content FrameworkA Reflection-Powered Infrastructure for Tools
Content Framework
Build System
Asset Repository
Asset Manager
Asset ExplorerPackage System
Content Framework
Reflection
Tool-side
Game-side
Asset Previewing
3rd Party
(Demo Videos)
• Fast tools development• Great workflows for all types of content• Decoupled systems with improved reusability and
resilience to change
Promising Results
Questions?