evolving multimodal networks for multitask games
DESCRIPTION
Evolving Multimodal Networks for Multitask Games. Jacob Schrum – [email protected] Risto Miikkulainen – [email protected] University of Texas at Austin Department of Computer Science. Evolution in videogames Automatically learn interesting behavior Complex but controlled environments - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Evolving Multimodal Networks for Multitask Games](https://reader035.vdocuments.site/reader035/viewer/2022062310/56815f6d550346895dce74b5/html5/thumbnails/1.jpg)
Evolving Multimodal Networks for Multitask GamesJacob Schrum – [email protected] Miikkulainen – [email protected] of Texas at AustinDepartment of Computer Science
![Page 2: Evolving Multimodal Networks for Multitask Games](https://reader035.vdocuments.site/reader035/viewer/2022062310/56815f6d550346895dce74b5/html5/thumbnails/2.jpg)
![Page 3: Evolving Multimodal Networks for Multitask Games](https://reader035.vdocuments.site/reader035/viewer/2022062310/56815f6d550346895dce74b5/html5/thumbnails/3.jpg)
Evolution in videogames Automatically learn interesting behavior Complex but controlled environments
Stepping stone to real world Robots Training simulators
Complexity issues Multiple contradictory objectives Multiple challenging tasks
![Page 4: Evolving Multimodal Networks for Multitask Games](https://reader035.vdocuments.site/reader035/viewer/2022062310/56815f6d550346895dce74b5/html5/thumbnails/4.jpg)
Multitask Games
NPCs perform two or more separate tasks Each task has own performance measures Task linkage
IndependentDependent
Not blended Inherently multiobjective
![Page 5: Evolving Multimodal Networks for Multitask Games](https://reader035.vdocuments.site/reader035/viewer/2022062310/56815f6d550346895dce74b5/html5/thumbnails/5.jpg)
Test Domains Designed to study multimodal behavior Two tasks in similar environments Different behavior needed to succeed Main challenge: perform well in both
Front Ramming Back Ramming
![Page 6: Evolving Multimodal Networks for Multitask Games](https://reader035.vdocuments.site/reader035/viewer/2022062310/56815f6d550346895dce74b5/html5/thumbnails/6.jpg)
Front/Back Ramming
Front Ramming Attack w/front ram Avoid counterattacks
Back Ramming Attack w/back ram Avoid counterattacks
Same goal, opposite embodiments
![Page 7: Evolving Multimodal Networks for Multitask Games](https://reader035.vdocuments.site/reader035/viewer/2022062310/56815f6d550346895dce74b5/html5/thumbnails/7.jpg)
Predator/Prey
Predator Attack prey Prevent escape
Prey Avoid attack Stay alive
Same embodiment, opposite goals
![Page 8: Evolving Multimodal Networks for Multitask Games](https://reader035.vdocuments.site/reader035/viewer/2022062310/56815f6d550346895dce74b5/html5/thumbnails/8.jpg)
Multiobjective Optimization Game with two objectives:
Damage Dealt Remaining Health
A dominates B iff A is strictly better in one objective and at least as good in others
Population of points not dominated are best: Pareto Front
Weighted-sum provably incapable of capturing non-convex front
Dealt lot of damage,but lost lots of health
Tradeoff between objectives
High health but did not deal much damage
![Page 9: Evolving Multimodal Networks for Multitask Games](https://reader035.vdocuments.site/reader035/viewer/2022062310/56815f6d550346895dce74b5/html5/thumbnails/9.jpg)
NSGA-II Evolution: natural approach for finding optimal population Non-Dominated Sorting Genetic Algorithm II*
Population P with size N; Evaluate P Use mutation to get P´ size N; Evaluate P´ Calculate non-dominated fronts of {P P´} size 2N New population size N from highest fronts of {P P´}
*K. Deb et al. A Fast and Elitist Multiobjective Genetic Algorithm: NSGA-II. Evol. Comp. 2002
![Page 10: Evolving Multimodal Networks for Multitask Games](https://reader035.vdocuments.site/reader035/viewer/2022062310/56815f6d550346895dce74b5/html5/thumbnails/10.jpg)
Constructive Neuroevolution Genetic Algorithms + Neural Networks Build structure incrementally (complexification) Good at generating control policies Three basic mutations (no crossover used)
Perturb WeightAdd Connection Add Node
![Page 11: Evolving Multimodal Networks for Multitask Games](https://reader035.vdocuments.site/reader035/viewer/2022062310/56815f6d550346895dce74b5/html5/thumbnails/11.jpg)
Multimodal Networks (1) Multitask Learning*
One mode per task Shared hidden layer Knows current task
Previous work Supervised learning context Multiple tasks learned
quicker than individual Not tried with evolution yet
* R. A. Caruana, "Multitask learning: A knowledge-based source of inductive bias" ICML 1993
![Page 12: Evolving Multimodal Networks for Multitask Games](https://reader035.vdocuments.site/reader035/viewer/2022062310/56815f6d550346895dce74b5/html5/thumbnails/12.jpg)
Multimodal Networks (2) Mode Mutation
Extra modes evolved Networks choose mode Chosen via preference neurons
MM Previous Links from previous mode Weights = 1.0
MM Random Links from random
sources Random weights Supports mode deletion
Starting network with one mode
MM(R)MM(P)
![Page 13: Evolving Multimodal Networks for Multitask Games](https://reader035.vdocuments.site/reader035/viewer/2022062310/56815f6d550346895dce74b5/html5/thumbnails/13.jpg)
Experiment Compare 4 conditions:
Control: Unimodal networks Multitask: One mode per task MM(P): Mode Mutation Previous MM(R): Mode Mutation Random + Delete Mutation
500 generations Population size 52 “Player” behavior scripted Network controls homogeneous team of 4
![Page 14: Evolving Multimodal Networks for Multitask Games](https://reader035.vdocuments.site/reader035/viewer/2022062310/56815f6d550346895dce74b5/html5/thumbnails/14.jpg)
MO Performance Assessment
Reduce Pareto front to single numberHypervolume of
dominated region Pareto compliant
Front A dominates front B implies HV(A) > HV(B)
Standard statistical comparisons of average HV
![Page 15: Evolving Multimodal Networks for Multitask Games](https://reader035.vdocuments.site/reader035/viewer/2022062310/56815f6d550346895dce74b5/html5/thumbnails/15.jpg)
20 runs
![Page 16: Evolving Multimodal Networks for Multitask Games](https://reader035.vdocuments.site/reader035/viewer/2022062310/56815f6d550346895dce74b5/html5/thumbnails/16.jpg)
Front/Back Ramming Behaviors
Multitask
MM(R)
Front Ramming Back Ramming
![Page 17: Evolving Multimodal Networks for Multitask Games](https://reader035.vdocuments.site/reader035/viewer/2022062310/56815f6d550346895dce74b5/html5/thumbnails/17.jpg)
20 runs
![Page 18: Evolving Multimodal Networks for Multitask Games](https://reader035.vdocuments.site/reader035/viewer/2022062310/56815f6d550346895dce74b5/html5/thumbnails/18.jpg)
Predator/Prey Behaviors
Multitask
MM(R)
Prey Predator
![Page 19: Evolving Multimodal Networks for Multitask Games](https://reader035.vdocuments.site/reader035/viewer/2022062310/56815f6d550346895dce74b5/html5/thumbnails/19.jpg)
Discussion (1) Front/Back Ramming
Control < MM(P), MM(R) < MultitaskMultiple modes helpExplicit knowledge of task helps
![Page 20: Evolving Multimodal Networks for Multitask Games](https://reader035.vdocuments.site/reader035/viewer/2022062310/56815f6d550346895dce74b5/html5/thumbnails/20.jpg)
Discussion (2) Predator/Prey
MM(P), Control, Multitask < MM(R)Multiple modes not necessarily helpfulDisparity in relative difficulty of tasks
Multitask ends up wasting effortMode deletion aids search for one good mode
![Page 21: Evolving Multimodal Networks for Multitask Games](https://reader035.vdocuments.site/reader035/viewer/2022062310/56815f6d550346895dce74b5/html5/thumbnails/21.jpg)
How To Apply Multitask good if:
Task division known, andTasks are comparably difficult
Mode mutation good if:Task division is unknown, or“Obvious” task division is misleading
![Page 22: Evolving Multimodal Networks for Multitask Games](https://reader035.vdocuments.site/reader035/viewer/2022062310/56815f6d550346895dce74b5/html5/thumbnails/22.jpg)
Future Work Games with more tasks
Does method scale? Control mode bloat
Games with independent tasks Ms. Pac-Man
Collect pills while avoiding ghosts Eat ghosts after eating power pill
Games with blended tasks Unreal Tournament 2004
Fight while avoiding damage Fight or run away? Collect items or seek opponents?
![Page 23: Evolving Multimodal Networks for Multitask Games](https://reader035.vdocuments.site/reader035/viewer/2022062310/56815f6d550346895dce74b5/html5/thumbnails/23.jpg)
Conclusion Domains with multiple tasks are common
Both in real world and games Multimodal networks improve learning in
multitask games Will allow interesting/complex behavior to
be developed in future
![Page 24: Evolving Multimodal Networks for Multitask Games](https://reader035.vdocuments.site/reader035/viewer/2022062310/56815f6d550346895dce74b5/html5/thumbnails/24.jpg)
Questions?Jacob Schrum – [email protected] Miikkulainen – [email protected]
University of Texas at AustinDepartment of Computer Science
![Page 25: Evolving Multimodal Networks for Multitask Games](https://reader035.vdocuments.site/reader035/viewer/2022062310/56815f6d550346895dce74b5/html5/thumbnails/25.jpg)
Auxiliary Slides
![Page 26: Evolving Multimodal Networks for Multitask Games](https://reader035.vdocuments.site/reader035/viewer/2022062310/56815f6d550346895dce74b5/html5/thumbnails/26.jpg)
![Page 27: Evolving Multimodal Networks for Multitask Games](https://reader035.vdocuments.site/reader035/viewer/2022062310/56815f6d550346895dce74b5/html5/thumbnails/27.jpg)