metagamer: an agent for learning and planning in general games barney pell nasa ames research center

20
METAGAMER: An Agent for Learning and Planning in General Games Barney Pell NASA Ames Research Center

Upload: olivia-davis

Post on 27-Mar-2015

212 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: METAGAMER: An Agent for Learning and Planning in General Games Barney Pell NASA Ames Research Center

METAGAMER: An Agent for Learning and

Planning in General Games Barney Pell

NASA Ames Research Center

Page 2: METAGAMER: An Agent for Learning and Planning in General Games Barney Pell NASA Ames Research Center

OUTLINE OF TALK

• METAGAME• Chess-Like Games and Generation• METAGAMER• Performance• Related Work• Implications for learning and reasoning in games• Conclusion

Page 3: METAGAMER: An Agent for Learning and Planning in General Games Barney Pell NASA Ames Research Center

Knight-Zone Chess

Page 4: METAGAMER: An Agent for Learning and Planning in General Games Barney Pell NASA Ames Research Center

META-GAME PLAYING

• Diverse Class of Games• Automated Game Designer• Uniform Representation• Programs Must Analyze Rules• No Existing Experts• Evaluation by Metagame Tournament• Increase challenge by extending class over time

Page 5: METAGAMER: An Agent for Learning and Planning in General Games Barney Pell NASA Ames Research Center

TOURNAMENT FORMAT

• Accept Rules• Initial Analysis• Individual Contests• Post-Mortem Analysis• Time Limits• No Programmer Modification• Winner

Page 6: METAGAMER: An Agent for Learning and Planning in General Games Barney Pell NASA Ames Research Center

Computer Game-Playing ResearchClass

of games

General game knowledge

Resource bounds

Game rules

Competitive context

Player

Specific game knowledge

OpponentPlayer

(Minimal use today)

(Heavy use today)

Page 7: METAGAMER: An Agent for Learning and Planning in General Games Barney Pell NASA Ames Research Center

Computer Game-Playing ResearchClass

of games

General game knowledge

Resource bounds

Game rules

Competitive context

Player

Specific game knowledge

OpponentPlayer

Page 8: METAGAMER: An Agent for Learning and Planning in General Games Barney Pell NASA Ames Research Center

Computer Game-Playing ResearchClass

of games

General game knowledge

Resource bounds

Game rules

Competitive context

Player

Specific game knowledge

OpponentPlayer

Page 9: METAGAMER: An Agent for Learning and Planning in General Games Barney Pell NASA Ames Research Center

Game rules

Game rules

Game generator

Metagamer

Opponent Metagamer

Meta-Game-Playing ResearchClass

of games

General game knowledge

Resource bounds

Game rules

Competitive context

Player

Specific game knowledge

OpponentPlayer

Page 10: METAGAMER: An Agent for Learning and Planning in General Games Barney Pell NASA Ames Research Center

Class and Generator

• Symmetric Chess-Like Games – Global Symmetry– Board

• Pieces• Initial Setup

• Goals– Includes many known games of varying complexity

• Game Generator– Stochastic Context-Free Generation– Controllable Parameters – Generates some interesting games

Page 11: METAGAMER: An Agent for Learning and Planning in General Games Barney Pell NASA Ames Research Center

METAGAMER

• Class and Strategy in General Representation• Game-Specializer: Compiles to Improve Efficiency• Game-Analyzer: Produces Specialized Analysis Tables• Advisors: Use Analysis Tables to Evaluate Position• Weights

– Relative Importance of General Advisors– Tuned by experiments– Values not as crucial as for base-level

• Search Engine: Alpha-Beta Minimax

Page 12: METAGAMER: An Agent for Learning and Planning in General Games Barney Pell NASA Ames Research Center

Advisors for Chess-Like Games

• Mobility– dynamic-mobility– static-mobility– capturing-mobility– eventual-mobility

• Threats and Capturing– global-threats– potent-threats– possession

• Goals and Step Functions– Vital– arrival-distance– promote-distance

Page 13: METAGAMER: An Agent for Learning and Planning in General Games Barney Pell NASA Ames Research Center
Page 14: METAGAMER: An Agent for Learning and Planning in General Games Barney Pell NASA Ames Research Center
Page 15: METAGAMER: An Agent for Learning and Planning in General Games Barney Pell NASA Ames Research Center

Results in Competition• Checkers

– Stronger than Greedy-Material– 1-man handicap ==> draws strong opponent– Strong if 1-man handicap

• Chess– Stronger than Greedy-Material– Can Defeat Human Novices– Good Positional Play, Weak Tactics

• Other games– Chinese chess, Japanese chess, Chess variations: “Sensible play”

• Generated Games (w/o human assistance)– All Advisors ==> won Tourney – No Version was best on every game– Knowledge outperforms Search (so far!)

• "Rediscovers" Known Strategies• Long-range strategic capabilities with limited search• Learning Gives Improvement

Page 16: METAGAMER: An Agent for Learning and Planning in General Games Barney Pell NASA Ames Research Center

Related work

• Other work in learning and planning games– Forks, abstraction, parameter-learning,

feature-learning and generation

• Metagamer works on unknown games• Does not rely on strong opponents• Benefits from Rules• Plays Entire Game

Page 17: METAGAMER: An Agent for Learning and Planning in General Games Barney Pell NASA Ames Research Center

Implications for learning and planning in general games

• Game analysis like scientific investigation• Intellectual development

– Discipline for perceiving, searching, reacting, time mgmt– Practice and training– Progression of skills

• Multi-strategy approaches– Constraint-based design– Theorems and lemmas– Analogies– Theory-driven experiments– Exploration and Trial and error

• Cultural– Transfer of knowledge– Authorship and history

Page 18: METAGAMER: An Agent for Learning and Planning in General Games Barney Pell NASA Ames Research Center

CONCLUSION• Metagame reveals wide open problems• Attractive properties as evaluation testbed

– Competitive performance criteria– Quantifiable demonstration of generality– Requires learning and reasoning on integrated problems– Humans have high competence, so impressive if programs could play well– Increasing challenges over time

• More general classes of problems (eg chess + go)• Larger scale problems (bigger boards, more pieces)• More complex domain attributes (eg multi-player, incomplete information, chance)

• Chess-Like Games is a good start– Existence proof that something is possible here– Hard problem (little improvement in 10 years!) – Workbench makes development easy

• Similar ideas could be applied to other challenges– Eg. planning, categorization, robotics competitions

• Key to any of these– Quantify claims of generality to the information available to humans in system– Removing information forces new challenges for agents

Page 19: METAGAMER: An Agent for Learning and Planning in General Games Barney Pell NASA Ames Research Center
Page 20: METAGAMER: An Agent for Learning and Planning in General Games Barney Pell NASA Ames Research Center