jonas schneider, head of engineering for robotics, openai
Post on 22-Jan-2018
565 Views
Preview:
TRANSCRIPT
OpenAI
Non-profit research lab
Goal: ensure AGI is good for humanity
Teams: Robotics, Dota, basic research, …
Robots that Learn
https://blog.openai.com/robots-that-learn/
Data munging
Compute infra Networking
Observability Tooling Regression tests
ML core (e.g. PPO, A3C, …)
Deployment/Inference
Storage
Orchestration
Data munging
Compute infra Networking
Observability Tooling Regression tests
ML core (e.g. PPO, A3C, …)
Deployment/Inference
Storage
Orchestration
Kubernetes
Azure
Kubernetes
GCE
Kubernetes
On-Premises Hardware
Our Model Our Model Our Model
Example: Orchestration
Scriptable infrastructure
exp = Experiment() exp.add_parameter_server()
for i in range(NUM_WORKERS): exp.add_tensorflow_worker(my_tf_graph, cpu=24, gpu=4)
exp.run(mode=’kube’) # or ’docker’
https://blog.openai.com/infrastructure-for-deep-learning/ “Building the Infrastructure that powers the future of AI”, KubeCon 2017
Think:
Instead of:Research Engineering
Systems
AlgorithmsTRPO
PPODQN
ES
?
https://blog.openai.com/evolution-strategies/ https://blog.openai.com/openai-baselines-ppo/
How to scale RL?
Supervised learning: gradient averaging
Large batch sizes fix many problems
Turns out, it works for reinforcement learning too
Know your stack
CUDA bindings
TF Graph Language
Distributed TF
Seems fast until you see PyTorch
Performance issues on plain Ethernet
Nice design, takes getting used to
top related