rl and deep learning - mlss2014.com · hierarchical reinforcement learning high-level model-based...

RL and deep learning Nando de Freitas

Upload: voduong

Post on 16-Feb-2019

220 views

Category:

Documents

0 download

Report

Download

Embed Size (px):

TRANSCRIPT

Page 1: RL and deep learning - mlss2014.com · Hierarchical reinforcement learning High-level model-based learning for deciding when to navigate, park, pickup and dropoff passengers. Mid-level

RL and deep learningNando de Freitas

Page 2: RL and deep learning - mlss2014.com · Hierarchical reinforcement learning High-level model-based learning for deciding when to navigate, park, pickup and dropoff passengers. Mid-level

Page 3: RL and deep learning - mlss2014.com · Hierarchical reinforcement learning High-level model-based learning for deciding when to navigate, park, pickup and dropoff passengers. Mid-level

Page 4: RL and deep learning - mlss2014.com · Hierarchical reinforcement learning High-level model-based learning for deciding when to navigate, park, pickup and dropoff passengers. Mid-level

Google’s neural net learns just by watching youtube videos

Page 5: RL and deep learning - mlss2014.com · Hierarchical reinforcement learning High-level model-based learning for deciding when to navigate, park, pickup and dropoff passengers. Mid-level

Place cells in the hippocampus

Page 6: RL and deep learning - mlss2014.com · Hierarchical reinforcement learning High-level model-based learning for deciding when to navigate, park, pickup and dropoff passengers. Mid-level

[Denil et al 2012]

Hierarchical reinforcement learning

High-level model-based learning for deciding when to navigate, park, pickup and dropoff passengers.

Mid-level active path learning for navigating a topological map.

Low-level active policy optimizer to learn control of continuous non-linear vehicle dynamics.

Page 8: RL and deep learning - mlss2014.com · Hierarchical reinforcement learning High-level model-based learning for deciding when to navigate, park, pickup and dropoff passengers. Mid-level

Active Path Finding in Middle LevelNavigate policy generates sequence of waypoints on a topological map to navigate from a location to a destination.

Page 9: RL and deep learning - mlss2014.com · Hierarchical reinforcement learning High-level model-based learning for deciding when to navigate, park, pickup and dropoff passengers. Mid-level

Low-Level: Trajectory following

VyYerr

err

TORCS: 3D game engine that implements complex vehicle dynamics complete with manual and automatic transmission, engine, clutch, tire, and suspension models.

Page 10: RL and deep learning - mlss2014.com · Hierarchical reinforcement learning High-level model-based learning for deciding when to navigate, park, pickup and dropoff passengers. Mid-level

Bayesian optimization was used to find the neural net low-level policy and the value function for the upper levels

Page 11: RL and deep learning - mlss2014.com · Hierarchical reinforcement learning High-level model-based learning for deciding when to navigate, park, pickup and dropoff passengers. Mid-level

Page 12: RL and deep learning - mlss2014.com · Hierarchical reinforcement learning High-level model-based learning for deciding when to navigate, park, pickup and dropoff passengers. Mid-level

Deepmind approach

Page 13: RL and deep learning - mlss2014.com · Hierarchical reinforcement learning High-level model-based learning for deciding when to navigate, park, pickup and dropoff passengers. Mid-level

Deepmind approach

Page 14: RL and deep learning - mlss2014.com · Hierarchical reinforcement learning High-level model-based learning for deciding when to navigate, park, pickup and dropoff passengers. Mid-level

Deepmind approach

Page 15: RL and deep learning - mlss2014.com · Hierarchical reinforcement learning High-level model-based learning for deciding when to navigate, park, pickup and dropoff passengers. Mid-level

Imitation learning & mirror neurons

Page 16: RL and deep learning - mlss2014.com · Hierarchical reinforcement learning High-level model-based learning for deciding when to navigate, park, pickup and dropoff passengers. Mid-level

Imitation learning for Atari

[Dejan, Miroslav, 2014]

Page 17: RL and deep learning - mlss2014.com · Hierarchical reinforcement learning High-level model-based learning for deciding when to navigate, park, pickup and dropoff passengers. Mid-level

Imitation learning for Atari

[Dejan Markovikj, Miroslav Bogdanovic, Misha Denil, NdF 2014]

Page 18: RL and deep learning - mlss2014.com · Hierarchical reinforcement learning High-level model-based learning for deciding when to navigate, park, pickup and dropoff passengers. Mid-level

Inverse RL…or teaching deepmind how to play Atari

Page 19: RL and deep learning - mlss2014.com · Hierarchical reinforcement learning High-level model-based learning for deciding when to navigate, park, pickup and dropoff passengers. Mid-level

Next lecture: scalable learning

Thank you

LEARNING OOOOUTCOMES VVVALIDATION …...Learning Outcomes Level 1 Level 2 Level 3 Level 4 Level 5 Level 6 Level 7 Level 8 Basic skills required to carry out simple tasks; Basic cognitive

school schematic with dropoff and adaag

2020-2021 Before & After School …...ing to the best of our ability. Dropoff (Early Bird, before school transporta-tion/care) • Dropoff begins at 6:30AM (no sooner) and will take

Learning Korean language intermediate level 2

ENTRY LEVEL FOUNDATION LEARNING - files.schudio.com

DROPOFF SCHOOL AM BUS # PICKUP TIME PM BUS # TIME …...2019-2020 bus stop list page 2 of 170 school am bus # pickup time pm bus # dropoff time # street ann smith elementary 18-077

Mail/Absentee Ballot Remote Dropoff Sites - Election Center Papers/Pinellas Co... · San Diego, California 2009 Professional Practices Program Mail/Absentee Ballot Remote Dropoff

· SWOT Analysis LETS Dropoff Current transit capacity in Livingston County is insufficient to meet the level of demand. All of Flint MTA's Regional Routes to Livingston County are

Introduction Blue Level English Learning

Eye level Learning Center manalapan

Course-Level Learning Objectives

UNIT OUTLINE - utas.edu.au · INTENDED LEARNING OUTCOMES RELATED ASSESSMENT CRITERIA OR MODULE LEVEL LEARNING OBJECTIVES ASSESSMENT METHODS COURSE LEVEL LEARNING OUTCOMES Learning

BEEKEEPING LEVEL-I Learning Guide

Learning Through Level Design: Using a learning taxonomy to map

Level-III Learning Guide-04

Nursery Level: Speaking & Learning Skills

DROPOFF and PICKUP INFORMATION

DROPOFF SCHOOL AM BUS # PICKUP TIME PM BUS # TIME # … · 2019-07-17 · 2019-2020 bus stop list page 1 of 120 school am bus # pickup time pm bus # dropoff time # street ann smith

Top level learning

Intermediate level Specification Learning and … Collective and Social Learning 15 ... Intermediate level Specification Learning and Development ... Intermediate level Specification

Level Up! Playing. Designing. Learning

District Level Online Learning

SCHOOL AM BUS# PICKUP PM BUS# DROPOFF ADDRESS … · 2018-07-30 · bus stop list_2018-2019.xlsx page 1 of 138 school am bus# pickup pm bus# dropoff address street academic options

Level descriptors and learning outcomes expressed in ...thelondonschool.it/wp-content/uploads/2017/09/level-descritors... · Level descriptors and learning outcomes expressed in language

Human-Level Machine Learning

Word-level information influences phonetic learning in ...cocosci.princeton.edu/tom/papers/LabPublications/WordInfoAdultsBabies.pdfWord-level information inﬂuences phonetic learning

DropOff Detection Blind - GitHub Pages

Oswego County FREE Tire Dropoff

Machine learning - A higher level of learning

Measuring Up on College- Level Learning on College-Level Learning on College-Level Learning Sponsored by the Pew Charitable Trusts Margaret Miller, Project

Full Answers for Level 2 Accounting Learning Workbook · Full Answers for Level 2 Accounting Learning Workbook ... for .. (.)

Learning low-level vision

Parent MS PM Lineup: Pickup- Dropoff

LEARNING OBJECTIVE MASING-MASING LEVEL

Level 3 Certificate in Supporting Teaching and Learning in ... · • Level 3 NVQ in Supporting Teaching and Learning in Schools • level 2 and level 3 qualifications in children’s