metagamer: an agent for learning and planning in general games barney pell nasa ames research center

METAGAMER: An Agent for Learning and

Planning in General Games Barney Pell

NASA Ames Research Center

OUTLINE OF TALK

• METAGAME• Chess-Like Games and Generation• METAGAMER• Performance• Related Work• Implications for learning and reasoning in games• Conclusion

Knight-Zone Chess

META-GAME PLAYING

• Diverse Class of Games• Automated Game Designer• Uniform Representation• Programs Must Analyze Rules• No Existing Experts• Evaluation by Metagame Tournament• Increase challenge by extending class over time

TOURNAMENT FORMAT

• Accept Rules• Initial Analysis• Individual Contests• Post-Mortem Analysis• Time Limits• No Programmer Modification• Winner

Computer Game-Playing ResearchClass

of games

General game knowledge

Resource bounds

Game rules

Competitive context

Player

Specific game knowledge

OpponentPlayer

(Minimal use today)

(Heavy use today)

Computer Game-Playing ResearchClass

of games

General game knowledge

Resource bounds

Game rules

Competitive context

Player

Specific game knowledge

OpponentPlayer

Computer Game-Playing ResearchClass

of games

General game knowledge

Resource bounds

Game rules

Competitive context

Player

Specific game knowledge

OpponentPlayer

Game rules

Game rules

Game generator

Metagamer

Opponent Metagamer

Meta-Game-Playing ResearchClass

of games

General game knowledge

Resource bounds

Game rules

Competitive context

Player

Specific game knowledge

OpponentPlayer

Class and Generator

• Symmetric Chess-Like Games – Global Symmetry– Board

• Pieces• Initial Setup

• Goals– Includes many known games of varying complexity

• Game Generator– Stochastic Context-Free Generation– Controllable Parameters – Generates some interesting games

METAGAMER

• Class and Strategy in General Representation• Game-Specializer: Compiles to Improve Efficiency• Game-Analyzer: Produces Specialized Analysis Tables• Advisors: Use Analysis Tables to Evaluate Position• Weights

– Relative Importance of General Advisors– Tuned by experiments– Values not as crucial as for base-level

• Search Engine: Alpha-Beta Minimax

Advisors for Chess-Like Games

• Mobility– dynamic-mobility– static-mobility– capturing-mobility– eventual-mobility

• Threats and Capturing– global-threats– potent-threats– possession

• Goals and Step Functions– Vital– arrival-distance– promote-distance

Results in Competition• Checkers

– Stronger than Greedy-Material– 1-man handicap ==> draws strong opponent– Strong if 1-man handicap

• Chess– Stronger than Greedy-Material– Can Defeat Human Novices– Good Positional Play, Weak Tactics

• Other games– Chinese chess, Japanese chess, Chess variations: “Sensible play”

• Generated Games (w/o human assistance)– All Advisors ==> won Tourney – No Version was best on every game– Knowledge outperforms Search (so far!)

• "Rediscovers" Known Strategies• Long-range strategic capabilities with limited search• Learning Gives Improvement

Related work

• Other work in learning and planning games– Forks, abstraction, parameter-learning,

feature-learning and generation

• Metagamer works on unknown games• Does not rely on strong opponents• Benefits from Rules• Plays Entire Game

Implications for learning and planning in general games

• Game analysis like scientific investigation• Intellectual development

– Discipline for perceiving, searching, reacting, time mgmt– Practice and training– Progression of skills

• Multi-strategy approaches– Constraint-based design– Theorems and lemmas– Analogies– Theory-driven experiments– Exploration and Trial and error

• Cultural– Transfer of knowledge– Authorship and history

CONCLUSION• Metagame reveals wide open problems• Attractive properties as evaluation testbed

– Competitive performance criteria– Quantifiable demonstration of generality– Requires learning and reasoning on integrated problems– Humans have high competence, so impressive if programs could play well– Increasing challenges over time

• More general classes of problems (eg chess + go)• Larger scale problems (bigger boards, more pieces)• More complex domain attributes (eg multi-player, incomplete information, chance)

• Chess-Like Games is a good start– Existence proof that something is possible here– Hard problem (little improvement in 10 years!) – Workbench makes development easy

• Similar ideas could be applied to other challenges– Eg. planning, categorization, robotics competitions

• Key to any of these– Quantify claims of generality to the information available to humans in system– Removing information forces new challenges for agents