metagamer: an agent for learning and planning in general games barney pell nasa ames research center
TRANSCRIPT
METAGAMER: An Agent for Learning and
Planning in General Games Barney Pell
NASA Ames Research Center
OUTLINE OF TALK
• METAGAME• Chess-Like Games and Generation• METAGAMER• Performance• Related Work• Implications for learning and reasoning in games• Conclusion
Knight-Zone Chess
META-GAME PLAYING
• Diverse Class of Games• Automated Game Designer• Uniform Representation• Programs Must Analyze Rules• No Existing Experts• Evaluation by Metagame Tournament• Increase challenge by extending class over time
TOURNAMENT FORMAT
• Accept Rules• Initial Analysis• Individual Contests• Post-Mortem Analysis• Time Limits• No Programmer Modification• Winner
Computer Game-Playing ResearchClass
of games
General game knowledge
Resource bounds
Game rules
Competitive context
Player
Specific game knowledge
OpponentPlayer
(Minimal use today)
(Heavy use today)
Computer Game-Playing ResearchClass
of games
General game knowledge
Resource bounds
Game rules
Competitive context
Player
Specific game knowledge
OpponentPlayer
Computer Game-Playing ResearchClass
of games
General game knowledge
Resource bounds
Game rules
Competitive context
Player
Specific game knowledge
OpponentPlayer
Game rules
Game rules
Game generator
Metagamer
Opponent Metagamer
Meta-Game-Playing ResearchClass
of games
General game knowledge
Resource bounds
Game rules
Competitive context
Player
Specific game knowledge
OpponentPlayer
Class and Generator
• Symmetric Chess-Like Games – Global Symmetry– Board
• Pieces• Initial Setup
• Goals– Includes many known games of varying complexity
• Game Generator– Stochastic Context-Free Generation– Controllable Parameters – Generates some interesting games
METAGAMER
• Class and Strategy in General Representation• Game-Specializer: Compiles to Improve Efficiency• Game-Analyzer: Produces Specialized Analysis Tables• Advisors: Use Analysis Tables to Evaluate Position• Weights
– Relative Importance of General Advisors– Tuned by experiments– Values not as crucial as for base-level
• Search Engine: Alpha-Beta Minimax
Advisors for Chess-Like Games
• Mobility– dynamic-mobility– static-mobility– capturing-mobility– eventual-mobility
• Threats and Capturing– global-threats– potent-threats– possession
• Goals and Step Functions– Vital– arrival-distance– promote-distance
Results in Competition• Checkers
– Stronger than Greedy-Material– 1-man handicap ==> draws strong opponent– Strong if 1-man handicap
• Chess– Stronger than Greedy-Material– Can Defeat Human Novices– Good Positional Play, Weak Tactics
• Other games– Chinese chess, Japanese chess, Chess variations: “Sensible play”
• Generated Games (w/o human assistance)– All Advisors ==> won Tourney – No Version was best on every game– Knowledge outperforms Search (so far!)
• "Rediscovers" Known Strategies• Long-range strategic capabilities with limited search• Learning Gives Improvement
Related work
• Other work in learning and planning games– Forks, abstraction, parameter-learning,
feature-learning and generation
• Metagamer works on unknown games• Does not rely on strong opponents• Benefits from Rules• Plays Entire Game
Implications for learning and planning in general games
• Game analysis like scientific investigation• Intellectual development
– Discipline for perceiving, searching, reacting, time mgmt– Practice and training– Progression of skills
• Multi-strategy approaches– Constraint-based design– Theorems and lemmas– Analogies– Theory-driven experiments– Exploration and Trial and error
• Cultural– Transfer of knowledge– Authorship and history
CONCLUSION• Metagame reveals wide open problems• Attractive properties as evaluation testbed
– Competitive performance criteria– Quantifiable demonstration of generality– Requires learning and reasoning on integrated problems– Humans have high competence, so impressive if programs could play well– Increasing challenges over time
• More general classes of problems (eg chess + go)• Larger scale problems (bigger boards, more pieces)• More complex domain attributes (eg multi-player, incomplete information, chance)
• Chess-Like Games is a good start– Existence proof that something is possible here– Hard problem (little improvement in 10 years!) – Workbench makes development easy
• Similar ideas could be applied to other challenges– Eg. planning, categorization, robotics competitions
• Key to any of these– Quantify claims of generality to the information available to humans in system– Removing information forces new challenges for agents