existing and possible applications of artificial intelligence in computer games ori liel

Existing and Possible Existing and Possible Applications of Artificial Applications of Artificial IntelliIntelligencegence inin Computer Computer

GaGammesesOri LielOri Liel

In this lectureIn this lecture

• A few words about the industryA few words about the industry

• Current AI technologies applied in Current AI technologies applied in gamesgames

• ““Fuzzy logic”Fuzzy logic”

• Soar Quake-botSoar Quake-bot

• Interesting projects and possible Interesting projects and possible future applicationsfuture applications

The computer (and console) The computer (and console) game industrygame industry

• Multi-Billion $ industryMulti-Billion $ industry• Growing fastGrowing fast• Main trends: Main trends:

– Increasing # of games with multi-player optionsIncreasing # of games with multi-player options– Movement towards console gamesMovement towards console games– Emerging trend - increasing investment in AI Emerging trend - increasing investment in AI

• Graphics and sound can’t make ground-breaking leapsGraphics and sound can’t make ground-breaking leaps• More CPU power available: More CPU power available:

– Hardware improvementsHardware improvements– Graphics cards carry the burdenGraphics cards carry the burden

Need for AI – different game Need for AI – different game genersgeners• ActionAction

QuakeQuake IGI Covert StrikeIGI Covert Strike

– NeedNeed Formidable opponents Formidable opponents– NeedNeed good Support characters good Support characters

• Adventure & RPGAdventure & RPG MistMist NeverWinter NightsNeverWinter Nights

– NeedNeed Believable NPC’s Believable NPC’s• Have beliefs, desires and intentionsHave beliefs, desires and intentions• Are able to formulate and carry out plansAre able to formulate and carry out plans

– New notion: plot managementNew notion: plot management

Ori Liel

Should be able to describe the generes pretty quickly.

Need for AI – cont’dNeed for AI – cont’d

• StrategyStrategy Civilization 3Civilization 3 WarCraft 3WarCraft 3

– NeedNeed enemies with ability to plan, react and learn enemies with ability to plan, react and learn

• ““God-games”God-games” SimCitySimCity PopulousPopulous

– NeedNeed Believable environments Believable environments

Need for AI – cont’d Need for AI – cont’d

• Why are agents suitable?Why are agents suitable?• For For allall the reasons which were the reasons which were

mentioned before:mentioned before:– First of all: First of all: agents as a natural metaphoragents as a natural metaphor– ModularityModularity– DecentralizationDecentralization– ChangeabilityChangeability– Open environmentsOpen environments

Current State of AI in gamesCurrent State of AI in games

• AI is not taken very seriously (yet…)AI is not taken very seriously (yet…)– Budget goes to graphicsBudget goes to graphics– Usually put off until late in developmentUsually put off until late in development– Not enough expertise in the industryNot enough expertise in the industry

• From an industrial point of viewFrom an industrial point of view AI is AI is a means, not an end.a means, not an end.

Ori Liel

This means that it is enough that the game appear reasonably intelligent. It doesn't matter if it's all scripted and the computer is really dumb as a rock. Graphics sell

Current AI technologies Current AI technologies applied in gamesapplied in games

Current technology - FSMCurrent technology - FSM• Finite State Machines:Finite State Machines:

– States & transformation rulesStates & transformation rules– Agent behavior:Agent behavior:

• deterministicdeterministic• reactivereactive

– Can be improved by adding intelligent-looking Can be improved by adding intelligent-looking scripted behaviorscripted behavior

Patrol Alert

Retreat Com bat

START STATE TRANSITIONCONDITION

END STATE

Patrol Sound Heard AlertPatrol Armed Unit Spotted AlertPatrol Fired On CombatAlert Unit Recognised &

Attitude Hostile &Unit Outnumbered

Combat

Combat SignificantlyOutnumbered

Retreat

Ori Liel

Mention that these FSMs are very much in use but they are rather uninteresting

Current technology – Current technology – FlockingFlocking• Flocking algorithmsFlocking algorithms

– Simulate realistic Simulate realistic movement in a groupmovement in a group

– Based on studies of flocks Based on studies of flocks of birds, and schools of fishof birds, and schools of fish

– Essence of algorithms:Essence of algorithms:• Maintain a minimum distance Maintain a minimum distance

from other agentsfrom other agents

• Move towards center of massMove towards center of mass

• Keep your alignment similar Keep your alignment similar to other agents’to other agents’

“follow the leader” flocking

Ori Liel

Taken from artificial life. Mention squad tactics

Current technology – othersCurrent technology – others

• Pursuit-evasion Pursuit-evasion algorithmsalgorithms– AI character in pursuitAI character in pursuit– AI character escapingAI character escaping

• Path-finding - A* Path-finding - A* algorithmalgorithm– Used by units to find Used by units to find

their way around in the their way around in the environmentenvironment

““Fuzzy” logicFuzzy” logic

(And it’s application in game (And it’s application in game AI)AI)

Fuzzy logicFuzzy logic

• Development of fuzzy logic:Development of fuzzy logic:– Plato said it first: there is something Plato said it first: there is something

between ‘true’ and ‘false’between ‘true’ and ‘false’– Since thenSince then

•Three-valued logic [-1, 0, 1]Three-valued logic [-1, 0, 1]

•N-valued logicN-valued logic

•Finally, infinite-valued logicFinally, infinite-valued logic

Fuzzy logic – cont’dFuzzy logic – cont’d

• The idea:The idea:– the universe is made of fuzzy sets.the universe is made of fuzzy sets.

•““The fuzzy set of old people”The fuzzy set of old people”

•““The fuzzy set of red objects”The fuzzy set of red objects”

– and a membership function ‘m’ and a membership function ‘m’

m: STATEMENT m: STATEMENT VALUE VALUE


• An example:An example:– Jane is 70 years old. Is Jane old? Jane is 70 years old. Is Jane old?

Depends on who you’re asking. Depends on who you’re asking. – Assume that membership in ‘old’ is Assume that membership in ‘old’ is

defined as follows: defined as follows:

mOLD(x) = {1 (if age(x)>100) }mOLD(x) = {1 (if age(x)>100) }

{0.01*age(x) {0.01*age(x) otherwise) otherwise) } }


• According to this definition:According to this definition: mOLD(Jane) = 0.7mOLD(Jane) = 0.7

– Question:Question:•What does this mean?What does this mean?

– Answer:Answer:• Jane’s degree of membership within the set Jane’s degree of membership within the set

of old people is 0.7of old people is 0.7

– Not to be confused with probabilistic Not to be confused with probabilistic approachapproach


• Basic definitions (A and B are sets):Basic definitions (A and B are sets): A is EMPTY A is EMPTY for all x, mA(x) = 0.0 for all x, mA(x) = 0.0 A = B A = B for all x: mA(x) = mB(x) for all x: mA(x) = mB(x) A’ : for all x: m(x)A' = 1 – m(x)A A’ : for all x: m(x)A' = 1 – m(x)A A is CONTAINED in B A is CONTAINED in B for all x: for all x:

mA(x) <= mB(x)mA(x) <= mB(x) C = A C = A ۷۷ B B mC(x) = MAX(mA(x), mB(x)) mC(x) = MAX(mA(x), mB(x)) C = A C = A ۸۸ B B mC(x) = MIN(mA(x), mB(x)) mC(x) = MIN(mA(x), mB(x))


• Examples of last two definitions:Examples of last two definitions:– mOLD(Jane) = 0.7 mOLD(Jane) = 0.7 – mSMART(Jane) = 0.6mSMART(Jane) = 0.6– What is the degree of membership of Jane in What is the degree of membership of Jane in

the ‘the ‘old-or-smartold-or-smart’ set? ’ set? • m[OLD m[OLD ۷۷ SMART](Jane) = 0.7SMART](Jane) = 0.7

– What is the degree of membership of Jane in What is the degree of membership of Jane in the ‘the ‘old-and-smartold-and-smart’ set? ’ set? • m[OLD m[OLD ۸۸ SMART](Jane) = 0.6SMART](Jane) = 0.6

Fuzzy logic – applicationsFuzzy logic – applications

• Fuzzy-logic introduces a Fuzzy-logic introduces a random element into random element into decision-makingdecision-making– Opponent’s personalities in Opponent’s personalities in

Civilization 3 are defined in Civilization 3 are defined in this way:this way:• m[aggressive] (Japan) = 0.7m[aggressive] (Japan) = 0.7• m[scientific](Japan) = 0.4m[scientific](Japan) = 0.4• m[expansionistic](Japan) = m[expansionistic](Japan) =

0.60.6– A probabilistic function uses A probabilistic function uses

these values (among these values (among others) to determine Japan’s others) to determine Japan’s next move, depending on next move, depending on Japan’s present state. Japan’s present state.

ALLIEDtreaty = 0.8assist = 0.8

COOPERATIVEtreaty = 0.4assist = 0.2

Neutraltreaty = 0.1attack = 0.1assist = 0.05

AGGRESIVEattack = 0.6

ceasefire = 0.3peace = 0.8

HOSTILEattack = 0.95

ceasefire = 0.05

goodwill > 30 or gain > -50

gain >10

goodwill > 45

30 < goodwill < 45or gain > 60

goodwill < 50 and gain > 90

goodwill < 85

goodwill < 65goodwill > 65

goodwill < 65

goodwill > 85

goodwill > 85

Probabilistic Fuzzy-State Probabilistic Fuzzy-State machinemachine

The ‘Soar’ Quake-botThe ‘Soar’ Quake-bot

Quake IIQuake II

• Quake II – a first Quake II – a first person shooterperson shooter– The Quake II The Quake II

environmentenvironment• Levels: sets of rooms Levels: sets of rooms

and passagesand passages

• Weapons: from knife Weapons: from knife to RPGto RPG

• Power-upsPower-ups

– Quake II Death-Quake II Death-MatchMatch

Quake II - modsQuake II - mods

• ‘‘modsmods’ are modified versions of existing ’ are modified versions of existing gamesgames

• New game-play featuresNew game-play features

• Altered laws of physics, changes to environmentAltered laws of physics, changes to environment

• Improved (or at least different) AIImproved (or at least different) AI

– Can be created by company or by players.Can be created by company or by players.– In Quake II, id software released some source In Quake II, id software released some source

code, and mods were created by playerscode, and mods were created by players– Favorite use of code: Favorite use of code:

The creation of “The creation of “Quake-botsQuake-bots” for death-matches” for death-matches..

Ori Liel

Company interest - keep the game alive!

Quake II - Quake-botsQuake II - Quake-bots

Status of the targets:

target In range visible In front

A Yes No Yes

B Yes No No

C Yes Yes Yes

D No N/A N/A

E Yes No No

Bots simulate human perception

The ‘Soar’ Quake-BotThe ‘Soar’ Quake-Bot

• The Soar bot’s decision cycle:The Soar bot’s decision cycle:

Ori Liel

Don't go into detail, that's what the next few slides are for

‘‘Soar’ Quake-Bot – cont’dSoar’ Quake-Bot – cont’d

• Evaluation, proposal and operator Evaluation, proposal and operator selectionselection take into account: take into account:

• Internal stateInternal state– Health levelHealth level– Current weaponCurrent weapon– Knowledge of mapKnowledge of map

•Some pre-defined ordering among Some pre-defined ordering among operators, which is unique to the version of operators, which is unique to the version of the robot. the robot.

‘‘Soar’ Quake-Bot – cont’dSoar’ Quake-Bot – cont’d

• Operator applicationOperator application– Operators can be:Operators can be:

•primitive actions (move, shoot)primitive actions (move, shoot)

• internal actions (remember location)internal actions (remember location)

•abstract actions (collect power-ups)abstract actions (collect power-ups)

– If an abstract operator has been If an abstract operator has been selected, it is immediately regarded as a selected, it is immediately regarded as a goalgoal

Get item

Goto next room

Get item in room

Detect item missing

Face item Move to item

Stop moving to item

Go through door

Go to door

Align with door

Face door

Move to door

Record at door

Slide to door

Stop move to door

Stop slide to door

Collect-Powerups

Choose-best-item

‘‘Soar’ Quake-Bot – Soar’ Quake-Bot – AnticipationAnticipation

• Robot anticipates player’s moves by:Robot anticipates player’s moves by:– Forming an internal model of playerForming an internal model of player

•Player statusPlayer status– HealthHealth– Weapon levelWeapon level

•Player’s beliefs (current perception of the Player’s beliefs (current perception of the world)world)

– Projecting its own behavior on playerProjecting its own behavior on player

‘‘Soar’ Quake-Bot – Anticipation Soar’ Quake-Bot – Anticipation (cont’d)(cont’d)• WhyWhy should anticipation be used? should anticipation be used?

– AmbushAmbush– Hunt (pursuit-evasion)Hunt (pursuit-evasion)– Deny power-upsDeny power-ups

• WhenWhen should anticipation be used? should anticipation be used?– Not all the timeNot all the time

• Requires lots of CPU for reasoning.Requires lots of CPU for reasoning.• May interfere with regular mechanismsMay interfere with regular mechanisms

– Should be used whenShould be used when• Sensing enemy (some knowledge of enemy state)Sensing enemy (some knowledge of enemy state)• Enemy is far away and not facing the botEnemy is far away and not facing the bot

His Distance: 1

My Distance: 1

His Distance: 2

My Distance: 2

His Distance: 3

My Distance: 1 (but hall)

His Distance: 4

My Distance: 0 Ambush!

* Bot remembers * Bot remembers successful successful sequences sequences (learning)(learning)

Ori Liel

Note that the bot has some learning capability

Other interesting projectsOther interesting projects

““Haunt 2” Haunt 2” http://winter.http://winter.eecseecs..umichumich..eduedu

/haunt/overview.html/haunt/overview.html

• SituationSituation: player is a ghost-like : player is a ghost-like energy creature trapped in a energy creature trapped in a house house

• GoalGoal: escape: escape• ProblemProblem: player is severely : player is severely

limited in manipulating the limited in manipulating the environmentenvironment

• Game-playGame-play: get house residents : get house residents (AI characters) to do things for (AI characters) to do things for you by:you by:– frighteningfrightening– enticingenticing– cajolingcajoling– ThreateningThreateningGame-play entirely based on Game-play entirely based on

interaction with AI characters!interaction with AI characters!

http://winter.eecs.umich.edu/haunt/overview.html







““Haunt 2” – cont’dHaunt 2” – cont’d

• AI characters must have:AI characters must have:– PersonalityPersonality– Goals and motivationsGoals and motivations– Very believable reactions to Very believable reactions to

environmentenvironment• Example of AI characters:Example of AI characters:

– Evil scientistEvil scientist• Driven by desire to capture Driven by desire to capture

ghostghost• Personality: Immune to fear, Personality: Immune to fear,

can’t stand coldcan’t stand cold– Lost hitchhikerLost hitchhiker

• Driven by curiosityDriven by curiosity• Personality: Easily frightened, Personality: Easily frightened,

physically strongphysically strong

Ori Liel

Wouldn't it be nice if ppl could program more characters (like mods)?

Cyphesis Cyphesis http://www.http://www.worldforgeworldforge.org/.org/wfwf

/dev/eng/servers//dev/eng/servers/cyphesiscyphesis

• Idea: characters in virtual worlds Idea: characters in virtual worlds alive, all NPCs have ambitionsalive, all NPCs have ambitions

• Between multi-player and single-Between multi-player and single-playerplayer

• Scripting and planning in useScripting and planning in use

• A thought: how to make it A thought: how to make it computationally feasible?computationally feasible?

http://www.worldforge.org/wf/dev/eng/servers/cyphesis






Kemal Enver’s project Kemal Enver’s project http://www.http://www.

multiagentmultiagent..tktk

• A variation of capture the flag with A variation of capture the flag with tank fightstank fights– A* path-findingA* path-finding– Reactive behaviorReactive behavior– CommunicationCommunication between agents between agents

http://www.multiagent.tk/




Idea for a gameIdea for a game

• Set in a fantasy D&D-like environmentSet in a fantasy D&D-like environment• (As in Cyphesis) World full of self-(As in Cyphesis) World full of self-

interested NPCs.interested NPCs.• (As in “Haunt 2”) AI cooperation is very (As in “Haunt 2”) AI cooperation is very

importantimportant• Achieve cooperation by:Achieve cooperation by:

– NegotiationNegotiation– ThreatsThreats– Building trust and friendshipBuilding trust and friendship

Idea for a projectIdea for a project

• Simplified case of last ideaSimplified case of last idea• Setting: a zooSetting: a zoo

– PathsPaths– CagesCages– ObstaclesObstacles– ExitsExits– Items: tranquilizer gun, riding horse, security passesItems: tranquilizer gun, riding horse, security passes

• Agents:Agents:– Players (BDI, communication skills)Players (BDI, communication skills)– Animals (reactive, subsumption architecture or FSM)Animals (reactive, subsumption architecture or FSM)– Guards?Guards?

Idea for a project – cont’dIdea for a project – cont’d

• Game-play:Game-play:– Zoo is set randomlyZoo is set randomly– Missions are assigned randomlyMissions are assigned randomly

• Release Tiger into the wildRelease Tiger into the wild• Move animal Bear from cage ‘a’ to cage ‘b’Move animal Bear from cage ‘a’ to cage ‘b’• Keep at least ‘n’ animals in their cages (maintenance task)Keep at least ‘n’ animals in their cages (maintenance task)• Protect the Monkey at all costs!Protect the Monkey at all costs!

• Agents have goals Agents have goals agents have tasks. So game can be agents have tasks. So game can be seen as task-oriented domainseen as task-oriented domain

– Players don’t know about each other’s missions (an Players don’t know about each other’s missions (an opportunity to incorporate learning!)opportunity to incorporate learning!)

– Run simulation and see how fast, if at all, human player Run simulation and see how fast, if at all, human player completes task.completes task.

existing and possible applications of artificial intelligence in computer games ori liel

Documents

ai graphics

ai contdstrategycivilization

fuzzy logic contdaccording

fuzzy sets

jane old

industrial point of

years old

console game industrymulti