Managing Complexity in the

Halo2 AIDamián IslaBungie Studios

Managing Scalabilityin the

Halo2 AIDamián IslaBungie Studios

Scalability? No! Scalability!

Why Scalable AI?

More is BETTER

The brute force approach to common sense: the more unique situations you can recognize and react to, the more common-sensical you are.

(e.g. Cyc, OpenMind projects)

Variation

3 Dimensions of Scalability

Variety

Volumedrive the warthog

Shoot the needler

melee attack

fight

perchhide

boarding

Strafe target

heretic gruntelite major

floodswarms

hunters

hell-jumpersbrute captain

Miranda sniper marine

DRAMA!

pacingstory

challenge

The Ultimate Goal

DesignerI want X to do Y whenever Z!

EngineerOkay. (To himself) Hmm … and I know exactly

where to put that …

DesignerI’m happy now. (Trundles back to design-bubble)

AI Design Requirements

• Transparency– Facilitate the ongoing narrative in the player’s head

• Coherence– Focused action, right priorities

• Directability– For the designer

• Workability– For the engineer

See [Butcher & Griesemer]

GDC 2002

Managing Scalable

Decision-Makingin the

Halo2 AIDamián IslaBungie Studios

Decision making

Level scripting

Character definition

Memory

Perception

Decision making

Level scripting

Character definition

Memory

Perception

BehaviorsBehavior: a program that takes

temporary control of the actor in order to get something done

Behaviors are action over time:• Move to a point• Orient• Shoot• Crouch• Play dialogue• Trigger “actions”

Behaviors have notion of relevancy and duration

e.g.,• Fight• Vehicle entry• Throw grenade• Find cover• Cover friend• Check body• Wake up a sleeping grunt• Etc.

Halo2: ~130 in total

Target Selection

Behavior

Perception & Knowledge Model

LookingAiming / Shooting

Moving

Animation, Physics, Sound

World

Talking

The Combat Cycle

Idle CombatSelf-

preservation

Search

Flight

Decision-Making

Behavior Tree (or DAG)

Good• Intuitive• Modular• Scalable

Bad• Never really that simple• Debugging is hard• Memory is hard

Root

Self-preservation

Engage

Search

Charge

Fight

Guard

Cover

Presearch

Uncover

Guard

Grenade

Investigate

Idle Guard

Retreat Flee

Vehicle fight

Vehicle strafe

MeleeRoot

Engage

Search Uncover

Decision routines

• Parent custom

• Child-competitive

Parent

Child 2

Child 3

Child 4

Child 1

?

Decision routines

Child-competitive strategies• Analog Relevancy• Binary Relevancy

– Prioritized-list (root, engage)

– Sequential

– Sequential-looping (search)

– Randomized

– One-off randomized (postcombat)

Design Principle #1 :

Given A, B or C, always choose D (all of the above)

Parent

Child 2

Child 3

Child 4

Child 1

?

Impulses

Problem: What happens (with a prioritized list) when the priority is not constant?

Unless the player is in vehicle, in

which case...

Charge

Fight

Vehicle entry

Engage

Charge

Fight

Vehicle entryEngage

Impulses

Solution: Separate alternative trigger conditions out into separate impulse

Two execution options• In-place• Redirect

Charge

Fight

Vehicle entry

Player vehicle entry impulse

Engage

Charge

Fight

Vehicle entry

Vehicle entry

Engage

Charge

Fight

Vehicle entry

Player vehicle entry impulse

Engage

Charge

Fight

Vehicle entry

Self-preserve on damage impulse

Engage

Root

Self-preserve

Charge

Fight

Vehicle entry

Self-preserve on damage impulse

Engage

Root

Self-preserve

Charge

Fight

Vehicle entry

Self-preserve on damage impulse

Engage

Root

Self-preserve

Charge

Fight

Vehicle entry

Player vehicle entry impulse

Engage

Design Principle #2 :

Formalize complexity

(don’t hide it)

ImpulsesThe other purpose of impulses:

HACKSOr, “localized code with ad-hoc

Functionality”

e.g. • crouch_on_danger impulse• dive impulse

Charge

Fight

Vehicle entry

Player vehicle entry impulse

Engage

Charge

Fight

Vehicle entry

Vehicle entry

Engage

Charge

Fight

Vehicle entry

Self-preserve on damage impulse

Engage

Root

Self-preserve

Charge

Fight

Vehicle entry

Self-preserve on damage impulse

Engage

Root

Self-preserve

Charge

Fight

Vehicle entry

Self-preserve on damage impulse

Engage

Root

Self-preserve

Design Principle #3: Embrace the

hackery

Behavior MetadataProblem: In determining relevancy, we check the same conditions over and

over and over and over – Actor’s vehicle status (infantry, driver, passenger)– Actor’s alert status (idle, in combat, visible target)

Solution: Use metadata to describe execution conditions– Force all behaviors to declare their execution conditions– Halo2: “conditions” bitvector compared to “actual state” bitvector

Acts as a behavior/impulse mask (modifying the basic structure of the tree itself)

Root

Self-preservation

Engage

Search

Charge

Fight

Guard

Grenade

Cover

Presearch

Uncover

Guard

Grenade

Vehicle

Investigate

Suppressing fire

Grenade

Postcombat

Shoot corpse

Check corpse

Idle Guard

Retreat Flee

Melee Root

Self-preservation

Engage

Search

Charge

Guard

Cover

Uncover

Guard

Investigate

Idle Guard

Vehicle fight

Vehicle strafe

Root

Engage

Search

Fight

Guard

Presearch

Grenade

Suppressing fire

Grenade

Idle Guard

Custom BehaviorsProblem: character-type specific behaviors

– Don’t want all characters evaluating behaviors that only one type can do– E.g. grunt’s “retreat from scary enemy” impulse

Solution: attach custom behaviors to tree

Self-preservation

Engage

Search

Charge

Fight

Guard

Grenade

Retreat Flee

Vehicle fight

Generic

Root

Self-preservation

Engage

Search

Charge

Fight

Guard

Grenade

Retreat Flee

Vehicle fight

Retreat on danger impulse

Retreat on scary enemy impulse

Deploy turret impulse

Grunt

Root

Design Principle #4 :

Variation from a stable

base

Stimulus BehaviorsProblem: rare event-driven behaviors tested for every tick

Root

Self-preservation

Engage

Search

Charge

Fight

Guard

Grenade

Cover

Guard

Grenade

Vehicle

Retreat Flee

Vehicle fight

The world events

Stimulus System (event

handlers)

actore.g., Grunts flee when their leader dies

Solution: Stimulus BehaviorsBehaviors or impulses dynamically and

asynchronously placed into tree at specified location

Root

Self-preservation

Engage

Search

Charge

Fight

Guard

Grenade

Cover

Guard

Grenade

Vehicle

Retreat Flee

Vehicle fight

The world events

Stimulus System (event

handlers)

actor

Leader dead retreat impulse

Root

Self-preservation

Engage

Search

Charge

Fight

Guard

Grenade

Cover

Guard

Grenade

Vehicle

Retreat Flee

Vehicle fight

The world events

Stimulus System (event

handlers)

actor

Leader dead retreat impulse

Root

Self-preservation

Engage

Search

Charge

Fight

Guard

Grenade

Cover

Guard

Grenade

Vehicle

Retreat Flee

Vehicle fight

The world events

Stimulus System (event

handlers)

actor

“Actor Died”

1 sec.

Joint Behaviors

“Sure, why not?”

“Hey, buddy you wanna do this?”

Blackboard

“Hey, anybody wanna do this?”

RememberDesign

Principle #1!(always choose D,

all of the above)

Behavior Summary

• Behavior DAG• Binary decision routines• Impulses• Metadata• Custom Behaviors• Stimulus Behaviors• Joint behaviors

The behavior tree is fundamentally a DYNAMIC structure!

Decision making

Level scripting

Character definition

Memory

Perception

Memory and MemoryProblem: Persistent behavior state is impossible (too much data to keep

around!)Solution: Only keep around state for behaviors that are running

Problem: What about state we need to keep around regardless of whether we’re running the behavior or not? (e.g. last grenade-throw time)

BehaviorState

Root state Engage state Charge state Melee state unused

Self-preservation

Engage

Search

Charge

Fight

Guard

Grenade

Cover

Presearch

Uncover

Guard

Grenade

Investigate

Suppressing fire

Grenade

Vehicle fight

Vehicle strafe

MeleeRoot

Behavior

Persistent per-behavior

Behavior-state(short-term)

Per-target

Per-target

Per-target

The Anatomy of Memory

Store memory as…• Behavior state (short-term)• Persistent per-behavior• Persistent per-target

Props are also• Perception caches• Our knowledge model

…memory is a complicated thing.

“Props”

Behavior

Persistent per-behavior

Behavior-state(short-term)

Per-target

Per-target

Per-target

The Anatomy of Memory

e.g. Search

• Aquire target 1• Lose sight of target 1• Search target 1

– Last known location inspected– New search point selected

• Aquire target 2– Search deactivated, short-term

state discarded• Target 2 killed• Switch to target 1• Search target 1

– Where do we search?– Existing search point used

RememberDesign

Principle #2(formalize complexity,

don’t hide it)

Decision making

Level scripting

Character definition

Memory

Perception

Designer Direction

• Character definition– Model, animation graph– Behavior parameters– Static control

• Level-scripting– Dynamic Control

Decision making

Level scripting

Character definition

Memory

Perception

Variety

Variation

Static Control: Variety

• Character types– Elites

– Grunts

– Marines

• Variants– Gold elite

– Red elite

– Honor guard elite

– Jetpack elite

• Each variant gets a .character definition file

A LOT of parameters to author and maintain!

64(variants) x

80 (behaviors/variant) x

3 (parameters/behavior) =

15,360

Static Control: Parameters• Character files divided into

parameter blocks– Vitality properties– Perception properties– Search properties– Weapon usage– Etc.

• Blocks can be instantiated or not

• Hierarchy: children characters instantiate only significant variations from parent

• Root of tree is the “generic” character, which provides “reasonable” values for all parameters

Remember DesignPrinciple #4!(Variation from a

stable base)

Generic character

Grunt

Heretic grunt

Grunt major

Marine Elite

Hell-jumper

Elite major

Jetpack elite

Elite ultra

Static Control: Styles

Styles are cool:

• Behavior mask

• Bias control parameters

• Superposable

(Another reason why Impulses are great)

Dynamic Control: Organization

Halo1: • “Encounters”

– Groups of actors– Groups of areas

Problem: moving actors from one encounter to the next was “architecturally discouraged”

Halo2:• “Squads”

– Groups of actors

• “Zones”– Groups of areas

• “Orders”– A mapping between the

two

Dynamic Control: Orders

The Metaphor:• “Take that hill!”• “Hole up in that bunker!”

Or

“Occupy this space and behave this way!”• Position direction• Behavior direction

• Blah blah blah

Order“Initial”

Order“Forward”

Order“Rout”

Orders and Behavior

Behavior direction• Vehicle behaviors enabled/disabled• Active camouflage enabled/disabled• Follow player enabled/diabled• Rules of engagement• Style

– Behavior mask

– Bias control parameter

“Alright, men, engage active camouflage and move forward cautiously. And don’t open fire until the Master Chief does!”

In summary

Character Definition

Squad Order

Area

Style

StyleDesign

Principle #5 : Work with your represent

ations

Recap

Design Principles:1. Always choose D

(all of the above)

2. Formalize complexity (don’t hide it)

3. Embrace the hackery

4. Variation from stable base

5. Work with your representations

Technical Takeaways

• Dynamic behavior tree• Binary relevancy• Behavior masking• Principled approach to memory• Take smarts out of the

behavior and put it in the knowledge model

Conclusions

• All systems need to “play ball” with the core decision mechanism

• Not looking for “smart” architecture, looking for expressive architecture

The Ultimate Goal

Designer

I want X to do Y whenever Z!

Damián Isladamiani@microsoft.com

Questions?