Game Intelligence: The Future

Game Intelligence: The FutureSimon M. LucasGame Intelligence GroupSchool of CS & EEUniversity of EssexMeet Adrianne from nVidia

Beautiful, but not very bright,

yet.Game Intelligence GroupMain Activity:General purpose intelligence for game agents2 Academic staff1 post-doc10 PhD studentsApproachesEvolutionReinforcement LearningMonte Carlo Tree Search

Conventional Game Tree SearchMinimax with alpha-beta pruning, transposition tablesWorks well when:A good heuristic value function is knownThe branching factor is modestE.g. Chess: Deep Blue, RybkaTree grows exponentially with search depth

GoMuch tougher for computersHigh branching factorNo good heuristic value function

Although progress has been steady, it will take many decades of research and development before world-championshipcalibre go programs exist. Jonathan Schaeffer, 2001MCTS Operation(fig from CadiaPlayer, Bjornsson and Finsson, IEEE T-CIAIG)Each iteration starts at the rootFollows tree policy to reach a leaf nodeThen perform a random roll-out from thereNode N is then added to treeValue of T back-propagated up tree

8Upper Confidence Bounds on Trees (UCT) Node Selection PolicyFrom Kocsis and Szepesvari (2006)Aim: optimal balance between exploration and exploitationConverges to optimal policy given infinite number of roll-outsOften not used in practice!

Sample MCTS Tree (fig from CadiaPlayer, Bjornsson and Finsson, IEEE T-CIAIG)

Learning Tree Policy and Roll-Out PolicyResults for Othello (IEEE CIG 2011)

Research LeadershipGrantsConference SeriesJournalConference Special SessionsCompetitionsSoftware Toolkits (e.g. WOX, featured in MSDN Magazine)Research GrantsAI Games Network (EPSRC, 2007 2010; with Imperial and Bradford)UCT for Games and Beyond (EPSRC, 2010 2014, joint with Imperial and Bradford; 1.5M total, 489k Essex)Plus IEEE T-CIAIG Editorial Assistant + Travel

IEEE Transactions on Computational Intelligence and AI in GamesPublished quarterly since March 2009Journal has made an excellent start

15CompetitionsMany, many competitionsMost recent: Ms Pac-Man versus Ghost Team: IEEE CEC 2011An interesting and fun AI challengeCovered by New Scientist, Slashdot etcBy Philipp Rohlfshagen and David Robles

SummaryAI + GamesFantastic field to work inThe BEST test-bed for general intelligenceMonte-Carlo Tree Search + Reinforcement Learning: very promising!Reasonable standard general game playing is already a reality for many gamesWithin the next 10 years well enjoy interacting with life-like AI charactersSample Games