toward object-oriented deep reinforcement...

41
Matthew Botvinick DeepMind, London UK Gatsby Computational Neuroscience Unit, UCL Toward object-oriented deep reinforcement learning

Upload: others

Post on 08-May-2020

6 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Toward object-oriented deep reinforcement learningalgonauts.csail.mit.edu/slides/Algonauts2019_Matt_Botvinick.pdfon episodic memory and meta- learning. Alongside their interest as

Matthew BotvinickDeepMind, London UKGatsby Computational Neuroscience Unit, UCL

Toward object-oriented deep reinforcement learning

Page 2: Toward object-oriented deep reinforcement learningalgonauts.csail.mit.edu/slides/Algonauts2019_Matt_Botvinick.pdfon episodic memory and meta- learning. Alongside their interest as

+1

+1

atari

Mnih et al, Nature (2015)

Page 3: Toward object-oriented deep reinforcement learningalgonauts.csail.mit.edu/slides/Algonauts2019_Matt_Botvinick.pdfon episodic memory and meta- learning. Alongside their interest as

+1

+1

Jaderberg et al., Science, 2019

Page 4: Toward object-oriented deep reinforcement learningalgonauts.csail.mit.edu/slides/Algonauts2019_Matt_Botvinick.pdfon episodic memory and meta- learning. Alongside their interest as

+1

+1

dqn convnet

Mnih et al, Nature (2015)

Page 5: Toward object-oriented deep reinforcement learningalgonauts.csail.mit.edu/slides/Algonauts2019_Matt_Botvinick.pdfon episodic memory and meta- learning. Alongside their interest as

+1

+1lake

Page 6: Toward object-oriented deep reinforcement learningalgonauts.csail.mit.edu/slides/Algonauts2019_Matt_Botvinick.pdfon episodic memory and meta- learning. Alongside their interest as

+1

+1

Page 7: Toward object-oriented deep reinforcement learningalgonauts.csail.mit.edu/slides/Algonauts2019_Matt_Botvinick.pdfon episodic memory and meta- learning. Alongside their interest as

+1

+1objects — pic

Page 8: Toward object-oriented deep reinforcement learningalgonauts.csail.mit.edu/slides/Algonauts2019_Matt_Botvinick.pdfon episodic memory and meta- learning. Alongside their interest as

+1

+1human objects

Kahneman et al., 1992

Page 9: Toward object-oriented deep reinforcement learningalgonauts.csail.mit.edu/slides/Algonauts2019_Matt_Botvinick.pdfon episodic memory and meta- learning. Alongside their interest as

+1

+1

Egly, Driver, and Rafal (1994); Moore, Yantis, and Vaughan (1998)

Automatic spread of attention

Page 10: Toward object-oriented deep reinforcement learningalgonauts.csail.mit.edu/slides/Algonauts2019_Matt_Botvinick.pdfon episodic memory and meta- learning. Alongside their interest as

+1

+1

Roelfsema et al. Nature, 1998

Page 11: Toward object-oriented deep reinforcement learningalgonauts.csail.mit.edu/slides/Algonauts2019_Matt_Botvinick.pdfon episodic memory and meta- learning. Alongside their interest as

+1

+1

LO??? (Kanwisher)

Malach et al., PNAS, 1995

Page 12: Toward object-oriented deep reinforcement learningalgonauts.csail.mit.edu/slides/Algonauts2019_Matt_Botvinick.pdfon episodic memory and meta- learning. Alongside their interest as

+1

+1

objects — pic AGAIN

Page 13: Toward object-oriented deep reinforcement learningalgonauts.csail.mit.edu/slides/Algonauts2019_Matt_Botvinick.pdfon episodic memory and meta- learning. Alongside their interest as

+1

+1diuk (cf?)

cf. Keramati et al., 2018; Cobo et al., 2013; Garnelo et al., 2016; Lazaro-Gradillo et al., 2019; Zambaldi, et al., 2018

Page 14: Toward object-oriented deep reinforcement learningalgonauts.csail.mit.edu/slides/Algonauts2019_Matt_Botvinick.pdfon episodic memory and meta- learning. Alongside their interest as
Page 15: Toward object-oriented deep reinforcement learningalgonauts.csail.mit.edu/slides/Algonauts2019_Matt_Botvinick.pdfon episodic memory and meta- learning. Alongside their interest as

+1

+1

Page 16: Toward object-oriented deep reinforcement learningalgonauts.csail.mit.edu/slides/Algonauts2019_Matt_Botvinick.pdfon episodic memory and meta- learning. Alongside their interest as

+1

+1

E.g., Girshick, 2015; He et al., 2017; Redmon & Farhadi, 2018

Page 17: Toward object-oriented deep reinforcement learningalgonauts.csail.mit.edu/slides/Algonauts2019_Matt_Botvinick.pdfon episodic memory and meta- learning. Alongside their interest as

+1

+1

Page 18: Toward object-oriented deep reinforcement learningalgonauts.csail.mit.edu/slides/Algonauts2019_Matt_Botvinick.pdfon episodic memory and meta- learning. Alongside their interest as

Alex Lerchner Chris Burgess Loic Matthey Klaus Greff

Nick Watters Irina Higgins Rishabh Kabra Malcolm Reynolds

Page 19: Toward object-oriented deep reinforcement learningalgonauts.csail.mit.edu/slides/Algonauts2019_Matt_Botvinick.pdfon episodic memory and meta- learning. Alongside their interest as
Page 20: Toward object-oriented deep reinforcement learningalgonauts.csail.mit.edu/slides/Algonauts2019_Matt_Botvinick.pdfon episodic memory and meta- learning. Alongside their interest as
Page 21: Toward object-oriented deep reinforcement learningalgonauts.csail.mit.edu/slides/Algonauts2019_Matt_Botvinick.pdfon episodic memory and meta- learning. Alongside their interest as

half refrigerator

Page 22: Toward object-oriented deep reinforcement learningalgonauts.csail.mit.edu/slides/Algonauts2019_Matt_Botvinick.pdfon episodic memory and meta- learning. Alongside their interest as

other half refrigerator

Page 23: Toward object-oriented deep reinforcement learningalgonauts.csail.mit.edu/slides/Algonauts2019_Matt_Botvinick.pdfon episodic memory and meta- learning. Alongside their interest as

+1

+1

objects — pic AGAIN

Page 24: Toward object-oriented deep reinforcement learningalgonauts.csail.mit.edu/slides/Algonauts2019_Matt_Botvinick.pdfon episodic memory and meta- learning. Alongside their interest as
Page 25: Toward object-oriented deep reinforcement learningalgonauts.csail.mit.edu/slides/Algonauts2019_Matt_Botvinick.pdfon episodic memory and meta- learning. Alongside their interest as
Page 26: Toward object-oriented deep reinforcement learningalgonauts.csail.mit.edu/slides/Algonauts2019_Matt_Botvinick.pdfon episodic memory and meta- learning. Alongside their interest as
Page 27: Toward object-oriented deep reinforcement learningalgonauts.csail.mit.edu/slides/Algonauts2019_Matt_Botvinick.pdfon episodic memory and meta- learning. Alongside their interest as
Page 28: Toward object-oriented deep reinforcement learningalgonauts.csail.mit.edu/slides/Algonauts2019_Matt_Botvinick.pdfon episodic memory and meta- learning. Alongside their interest as
Page 29: Toward object-oriented deep reinforcement learningalgonauts.csail.mit.edu/slides/Algonauts2019_Matt_Botvinick.pdfon episodic memory and meta- learning. Alongside their interest as
Page 30: Toward object-oriented deep reinforcement learningalgonauts.csail.mit.edu/slides/Algonauts2019_Matt_Botvinick.pdfon episodic memory and meta- learning. Alongside their interest as

+1

Page 31: Toward object-oriented deep reinforcement learningalgonauts.csail.mit.edu/slides/Algonauts2019_Matt_Botvinick.pdfon episodic memory and meta- learning. Alongside their interest as

+1

+1

Page 32: Toward object-oriented deep reinforcement learningalgonauts.csail.mit.edu/slides/Algonauts2019_Matt_Botvinick.pdfon episodic memory and meta- learning. Alongside their interest as

+1

+1

Page 33: Toward object-oriented deep reinforcement learningalgonauts.csail.mit.edu/slides/Algonauts2019_Matt_Botvinick.pdfon episodic memory and meta- learning. Alongside their interest as

+1

+1

Page 34: Toward object-oriented deep reinforcement learningalgonauts.csail.mit.edu/slides/Algonauts2019_Matt_Botvinick.pdfon episodic memory and meta- learning. Alongside their interest as

+1

+1

Kahneman & Treisman, 1984: Object Files

Green, Edwin James, and Jake Quilty-Dunn. "what is an object file?." The British Journal for the Philosophy of Science (2017).

Page 35: Toward object-oriented deep reinforcement learningalgonauts.csail.mit.edu/slides/Algonauts2019_Matt_Botvinick.pdfon episodic memory and meta- learning. Alongside their interest as

+1

+1

Page 36: Toward object-oriented deep reinforcement learningalgonauts.csail.mit.edu/slides/Algonauts2019_Matt_Botvinick.pdfon episodic memory and meta- learning. Alongside their interest as

+1

+1

Page 37: Toward object-oriented deep reinforcement learningalgonauts.csail.mit.edu/slides/Algonauts2019_Matt_Botvinick.pdfon episodic memory and meta- learning. Alongside their interest as

+1

+1

Page 38: Toward object-oriented deep reinforcement learningalgonauts.csail.mit.edu/slides/Algonauts2019_Matt_Botvinick.pdfon episodic memory and meta- learning. Alongside their interest as

+1

+1

Page 39: Toward object-oriented deep reinforcement learningalgonauts.csail.mit.edu/slides/Algonauts2019_Matt_Botvinick.pdfon episodic memory and meta- learning. Alongside their interest as

+1

+1

Page 40: Toward object-oriented deep reinforcement learningalgonauts.csail.mit.edu/slides/Algonauts2019_Matt_Botvinick.pdfon episodic memory and meta- learning. Alongside their interest as

+1

+1

Page 41: Toward object-oriented deep reinforcement learningalgonauts.csail.mit.edu/slides/Algonauts2019_Matt_Botvinick.pdfon episodic memory and meta- learning. Alongside their interest as

Alex Lerchner Chris Burgess Loic Matthey Klaus Greff

Nick Watters Irina Higgins Rishabh Kabra Malcolm Reynolds