Gaming TechnologiesGaming Technologies

Craig PeeperSoftware ArchitectWindows Graphics & Gaming TechnologiesMicrosoft Corporation

Craig PeeperSoftware ArchitectWindows Graphics & Gaming TechnologiesMicrosoft Corporation

OverviewOverview

Games Yesterday & Today

Game Components

PC Platform & WGF 2.0

Game Trends

Big Challenges in Game Creation

Summary

Games Yesterday & Today

Game Components

PC Platform & WGF 2.0

Game Trends

Big Challenges in Game Creation

Summary

Games Yesterday & TodayGames Yesterday & Today

Quake 2 - 1997

Far Cry - 2004

Prince of Persia - 1989

Age of Empires IIIMax Payne - 2001

Doom - 1993

Platform

Game ComponentsGame Components

Content + Engine

Content ComponentsContent Components

story

scripting

animationtexturing /surfacing

lighting

Content

as well as …

• user interface

modeling

effects

leveldesign

sound

visualtechnologies

Content CreationContent Creation

Tool driven

Combination of tools forms “Art Pipeline”

Character Pipeline3D Model/Sculpting

Animation/Rigging

Skinning

Texturing

Lighting/Shading

An artist often specializes on one part of pipeline

Tool driven

Combination of tools forms “Art Pipeline”

Character Pipeline3D Model/Sculpting

Animation/Rigging

Skinning

Texturing

Lighting/Shading

An artist often specializes on one part of pipeline

Game Engine ComponentsGame Engine Components

AI

Physics

Audio

Graphics

Networking+

I/O

Engine as well as …

• installation

• patching

• resource management

• multithreading

• disk i/o

• state save/restore

Game EnginesGame Engines

Studio chooses to build or buyQuake Source, Unreal enginesRenderware, Gamebryo middlewareOften come with great authoring tools (level editors, etc.)

Componentized softwareMay buy specialized components

How many people can write a physics engine?Video codecs, etc

Engine may be optimized for game genre/styleEngine bounds range of content

Global illumination, terrain, indoor scenes

Studio chooses to build or buyQuake Source, Unreal enginesRenderware, Gamebryo middlewareOften come with great authoring tools (level editors, etc.)

Componentized softwareMay buy specialized components

How many people can write a physics engine?Video codecs, etc

Engine may be optimized for game genre/styleEngine bounds range of content

Global illumination, terrain, indoor scenes

Game Platform ComponentsGame Platform Components

CPU+

Memory

Storage

Audio

Graphics

Networking+

Input

SW + HWPlatform

Game PlatformsGame Platforms

Console10M – 100M installed base

“fixed platform”

Moderately powerful hardware

Standard & High definition video resolution (1080i, 720p)

Controller, wheel (& keyboard, mouse) input

Refreshed every 4-5 years

10’ experience on TV

Console10M – 100M installed base

“fixed platform”

Moderately powerful hardware

Standard & High definition video resolution (1080i, 720p)

Controller, wheel (& keyboard, mouse) input

Refreshed every 4-5 years

10’ experience on TV

Game Platforms (cont.)Game Platforms (cont.)

PC600M installed base (running Microsoft Windows)

Non-uniform platform (video cards, memory, CPU)

Most powerful hardwareLeast powerful too

Very high resolution (1600x1200, 1920x1080, …)

Keyboard, mouse (& controller) input

2’ & 10’ (desk vs living room)

Notebook (portable gaming)

Upgraded yearly!

PC600M installed base (running Microsoft Windows)

Non-uniform platform (video cards, memory, CPU)

Most powerful hardwareLeast powerful too

Very high resolution (1600x1200, 1920x1080, …)

Keyboard, mouse (& controller) input

2’ & 10’ (desk vs living room)

Notebook (portable gaming)

Upgraded yearly!

Other PlatformsOther Platforms

Hand-heldDedicated consoles (PSP, Gameboy)

Controller-like input

Mobile phones700M phones/year

Non-uniform platform

Keypad input

Modest resolution (320x200 … 640x480)

1995-class PC graphics

ArcadeIndustrial version of home console?

Hand-heldDedicated consoles (PSP, Gameboy)

Controller-like input

Mobile phones700M phones/year

Non-uniform platform

Keypad input

Modest resolution (320x200 … 640x480)

1995-class PC graphics

ArcadeIndustrial version of home console?

PC as a PlatformPC as a Platform

Large installed baseLeast expensive to publish on

No platform royalties

Large hw variation, rapidly evolvingLarge number of support calls

Supports rapid innovationProfit from selling PC hardware!

Games must span variabilityInfluences content & engineReach versus development cost

ChallengesAbstract hardware differences in platformSupport innovation

Large installed baseLeast expensive to publish on

No platform royalties

Large hw variation, rapidly evolvingLarge number of support calls

Supports rapid innovationProfit from selling PC hardware!

Games must span variabilityInfluences content & engineReach versus development cost

ChallengesAbstract hardware differences in platformSupport innovation

New Direction for PC PlatformNew Direction for PC Platform

Reduce “unnecessary” hardware variationProgrammable hardware causes new problems

Shading programs are part of content not engineShading programs more complex

Content must be portable & reusable

Approach:Specify details that matter

Representation, precision, accuracy

Don’t specify non-observable implementation details

Use virtual machine model

Reduce “unnecessary” hardware variationProgrammable hardware causes new problems

Shading programs are part of content not engineShading programs more complex

Content must be portable & reusable

Approach:Specify details that matter

Representation, precision, accuracy

Don’t specify non-observable implementation details

Use virtual machine model

Windows Graphics Foundation 2.0Windows Graphics Foundation 2.0

Maintain consistent feature setEliminate capability bits (optional features)

Well defined behaviorMultiple implementations produce same result

E.g., IEEE-754 arithmetic

Migrate legacy fixed functions to shadersFixed-function lighting, alpha test, etc

Consistent shader programming modelVertex and pixel shaders use common instruction set

Obsolete assembly level programming

Add optimizations where large benefitScenario-specific formats (high dynamic range, normal maps)

Maintain consistent feature setEliminate capability bits (optional features)

Well defined behaviorMultiple implementations produce same result

E.g., IEEE-754 arithmetic

Migrate legacy fixed functions to shadersFixed-function lighting, alpha test, etc

Consistent shader programming modelVertex and pixel shaders use common instruction set

Obsolete assembly level programming

Add optimizations where large benefitScenario-specific formats (high dynamic range, normal maps)

Innovation in WGF 2.0Innovation in WGF 2.0

Integer & floating-point processing

More general data flowStream out to memory after vertex processing

Unify textures, render targets, vertex buffers

“Arrayed” resourcesSelect array element to render to in shader

Enables single-pass render to cube map

New pipeline stage (geometry shader)Whole-primitive processing

Integer & floating-point processing

More general data flowStream out to memory after vertex processing

Unify textures, render targets, vertex buffers

“Arrayed” resourcesSelect array element to render to in shader

Enables single-pass render to cube map

New pipeline stage (geometry shader)Whole-primitive processing

Geometry ShaderGeometry Shader

Shader stage that “sees” whole primitive3 vertices of a triangle (or 2 vertices of a line)As well as adjacency (6 vertices per triangle)

Can compute per-primitive dataPass to pixel shaderBarycentric parametersPlane equations

Can amplify dataExtrude edges, expand points, generate shellsFind silhouette edgesLimited amplification (not a general tessellator)

Shader stage that “sees” whole primitive3 vertices of a triangle (or 2 vertices of a line)As well as adjacency (6 vertices per triangle)

Can compute per-primitive dataPass to pixel shaderBarycentric parametersPlane equations

Can amplify dataExtrude edges, expand points, generate shellsFind silhouette edgesLimited amplification (not a general tessellator)

WGF 2.0 PipelineWGF 2.0 Pipeline

VertexBuffer

InputAssembler

VertexShader

SetupRasterizer

OutputMerger

PixelShader

GeometryShader

IndexBuffer

Texture TextureRenderTarget

DepthStencil

TextureStreamBuffer

Stream out

Memory

memory

programmable

fixed

Sampler Sampler Sampler

Constant Constant Constant

Game TrendsGame Trends

2004-2005 Record sales

Visually stunning titlesHalf Life 2, Halo 2, Far Cry, Doom 3

Largest worlds

Most detailed levels of realismNot just graphics: AI, physics, animation

Longest development times

Largest development costs

Bulk of cost in producing contentIncreasing ratio of artists : programmers

2004-2005 Record sales

Visually stunning titlesHalf Life 2, Halo 2, Far Cry, Doom 3

Largest worlds

Most detailed levels of realismNot just graphics: AI, physics, animation

Longest development times

Largest development costs

Bulk of cost in producing contentIncreasing ratio of artists : programmers

Trends - SoftwareTrends - Software

Demand for larger worlds

Demand for more detailFloppies vs CDs vs DVDs → HALO2 4.2GB

HD-DVD/Blueray → 20GB

Rising development costContent creation is the bottleneck

$10M content budget

Art Pipeline is not scaling

Amortize cost over multiple platforms

Demand for larger worlds

Demand for more detailFloppies vs CDs vs DVDs → HALO2 4.2GB

HD-DVD/Blueray → 20GB

Rising development costContent creation is the bottleneck

$10M content budget

Art Pipeline is not scaling

Amortize cost over multiple platforms

Trends – HardwareTrends – Hardware

CPU performanceClock speed brick wall

Transition to “multi-core” CPUs

Games are CPU-limitedGames are tuned until they are not GPU-limited

PC & consoles

GPU performance2x increase every 2 years

CPU performanceClock speed brick wall

Transition to “multi-core” CPUs

Games are CPU-limitedGames are tuned until they are not GPU-limited

PC & consoles

GPU performance2x increase every 2 years

HardwareLots of Room To Grow?HardwareLots of Room To Grow?

Enthusiast GPU today (soon):

1-2 Teraflops

110-90nm

~300-350M transistors

~80 Watts

Increasing attention onpower

efficiency

Enthusiast GPU today (soon):

1-2 Teraflops

110-90nm

~300-350M transistors

~80 Watts

Increasing attention onpower

efficiency

Game Development ChallengesGame Development Challenges

Creating & Managing DetailScalable process for content creation

CPU overloadMulti-core not a panacea

Volume of dataEffective run-time management of data

Tool power & expressivenessTraditional tools not powerful enough

Creating & Managing DetailScalable process for content creation

CPU overloadMulti-core not a panacea

Volume of dataEffective run-time management of data

Tool power & expressivenessTraditional tools not powerful enough

GPU as SolutionGPU as Solution

How best to use performance increases?

CPU bottlenecksOffload more processing to the GPU?

Simulations, physics?

ContentSupport greater detail

Don’t create more work for artists

Procedural detail?

How best to use performance increases?

CPU bottlenecksOffload more processing to the GPU?

Simulations, physics?

ContentSupport greater detail

Don’t create more work for artists

Procedural detail?

Procedural DetailProcedural Detail

Techniques used in film production

Scene Composition & Post ProcessingTranslucency & Antialiasing

Filter & Blend

Texture DetailProcedural generation

Modeling & AnimationTessellation + Displacement/Normal Maps

Techniques used in film production

Scene Composition & Post ProcessingTranslucency & Antialiasing

Filter & Blend

Texture DetailProcedural generation

Modeling & AnimationTessellation + Displacement/Normal Maps

Scene Composition & PostScene Composition & Post

“Post” effects becoming common placeHDR, bloom, tints, grain, motion blur, depth of field ….

Global atmosphereDon’t merge into individual assets

Next step, on the flyApply to individual scene elements

Composite scene elements together

Use to “break up” the scene – detailing

“Post” effects becoming common placeHDR, bloom, tints, grain, motion blur, depth of field ….

Global atmosphereDon’t merge into individual assets

Next step, on the flyApply to individual scene elements

Composite scene elements together

Use to “break up” the scene – detailing

Transparency Transparency

Traditionally render back to front with blendSort on CPU

Interactions with other algorithms (e.g., shadows)

Different algorithms forObjects with feathered edges (trees)

Transparent objects (windows)

Volumetric effects (particle systems)

Opacity versus CoverageUse alpha to represent both

Can we solve with hardware?

Traditionally render back to front with blendSort on CPU

Interactions with other algorithms (e.g., shadows)

Different algorithms forObjects with feathered edges (trees)

Transparent objects (windows)

Volumetric effects (particle systems)

Opacity versus CoverageUse alpha to represent both

Can we solve with hardware?

Unsorted TransparencyUnsorted Transparency

Traditionally solved with a-buffer algorithm

Save all of the pixel fragments for each pixel

Resolve at end of frame

Arbitrary number of fragments per pixel Lists (linked), dynamic allocation

Look at “good enough” solutionsSave some maximum number of fragments

Fixed-function or programmable solution?Fixed-function more efficient, but extra hardware

Traditionally solved with a-buffer algorithm

Save all of the pixel fragments for each pixel

Resolve at end of frame

Arbitrary number of fragments per pixel Lists (linked), dynamic allocation

Look at “good enough” solutionsSave some maximum number of fragments

Fixed-function or programmable solution?Fixed-function more efficient, but extra hardware

Texture Maps Texture Maps

Resolution increasing rapidly4Kx4K high-resolution maps

Nearing the 8Kx8K used for film

Too much work to paint this much detailDon’t paint individual skin pores

Digital camera capture not the answer

4Kx4K RGBA = 32MB @ 2x compression

Resolution increasing rapidly4Kx4K high-resolution maps

Nearing the 8Kx8K used for film

Too much work to paint this much detailDon’t paint individual skin pores

Digital camera capture not the answer

4Kx4K RGBA = 32MB @ 2x compression

Texture MapsTexture Maps

GPU virtual memory helps with size, but…Can we ever really store enough resolution?

A better way?Use procedural texture techniques

Shaders now expressive enoughDo they have enough performance?

Artists concentrate on formAlgorithmically add detail

“paint” algorithm ids

GPU virtual memory helps with size, but…Can we ever really store enough resolution?

A better way?Use procedural texture techniques

Shaders now expressive enoughDo they have enough performance?

Artists concentrate on formAlgorithmically add detail

“paint” algorithm ids

Surfaces and TessellationSurfaces and Tessellation

Increasing character complexity2K → 5K → 15K triangles/character

Increased skinning and morphing complexityk weights for each vertexHand-tuned for animation

Silhouette edges still look poorCompared to lighting and shading quality

Alternative – use higher-order surfacesBeziers, Catmull-Clark subdivision, NURBS, …

Many studios already model with surfacesConvert to triangles when packaging content

Increasing character complexity2K → 5K → 15K triangles/character

Increased skinning and morphing complexityk weights for each vertexHand-tuned for animation

Silhouette edges still look poorCompared to lighting and shading quality

Alternative – use higher-order surfacesBeziers, Catmull-Clark subdivision, NURBS, …

Many studios already model with surfacesConvert to triangles when packaging content

Tessellation AdvantagesTessellation Advantages

Animate and skin the control meshLess computation than per-vertex

Tessellate the resulting control mesh

Combine with displacement mappingUse for additional geometric detail

Adds detail to silhouette edges

Use normal maps for fine shading detail

Combine with adaptive tessellation?

Animate and skin the control meshLess computation than per-vertex

Tessellate the resulting control mesh

Combine with displacement mappingUse for additional geometric detail

Adds detail to silhouette edges

Use normal maps for fine shading detail

Combine with adaptive tessellation?

Skinned Control MeshSkinned Control Mesh

SummarySummary

Games increasingly more complexWorld size, detail

Bulk of cost in producing contentContent creation process not scaling

Amortize cost over multiple platforms

Try to reduce cost of additional detailRun-time generation of detail

Improved tools for controlling detail

Good areas for additional research

Games increasingly more complexWorld size, detail

Bulk of cost in producing contentContent creation process not scaling

Amortize cost over multiple platforms

Try to reduce cost of additional detailRun-time generation of detail

Improved tools for controlling detail

Good areas for additional research

Games TomorrowGames Tomorrow

QuestionsQuestions