“curious places” october, 2007 key centre of design computing and cognition, university of...
TRANSCRIPT
“Curious Places”
October, 2007
Key Centre of Design Computing and Cognition, University of Sydney
A Room that Adapts using Curiosity and
Supervised LearningKathryn Merrick, Mary Lou
Maher, Rob Saunders
Overview
Adaptable, Intelligent Environments
Curious Supervised Learning
A Curious, Virtual, Sentient Room
Limitations and Future Work
The computer for the 21st century Hundreds of computers per room Computers come and go (Weiser, 1991)
Adaptability is important at two levels: The middleware level The behaviour level
Adaptable, Intelligent Environments
Adaptable Middleware
Resource management and communication
Adaptability has been widely considered at this level Real time interaction Presence services Ad hoc networking
Intelligent Room Project
Gaia
BLIP Systems
Adaptable Behaviour
Adapting behaviour to human activities Supervised Learning The “Neural Network House” Data mining Considered in fixed domains
How can we achieve adaptive behaviour in response to changing hardware or software?
Adaptability by Curiosity and Learning
Curiosity adapts focus of attention to relevant learning goals
Learning adapts behaviour to fulfil goals Curious reinforcement learning Curious supervised learning
MyS
QL
Da
tab
ase
Projector
Rear project
ion screen
PC
Bluetooth blip nodes
Agent
Agent
Curious Information
Display
Curious Research
Space
Supervised Learning
“Learning from examples”
A supervised learning problem P can be represented formally by: A set S of sensed states A set A of actions A set X of examples Xi = (Si, Ai)
A policy π : S A
Complex, Dynamic Environments
Contain multiple learning problems P = {P1, P2, P3…}
Learning problems in P may change over time Addition of new problems Removal of obsolete problems
Aim to focus attention on states, actions and examples from a subset of problems Works by filtering
Identify potential tasks to learn or act upon
Compute curiosity values Arbitrate on what to filter
High curiosity may trigger learning or action
Low curiosity does not
Modelling Curiosity for Supervised Learning
S(t), X(t)
S(t)X(t)
Curiosity
Learning Action
Observations and events
Task Selection
Curiosity Value
Arbitration
The Curious Supervised Learning Agent
Past states, examples and actions are stored in an experience trajectory Y
Experiences may influence curiosity A(t)
S(t)
S(t), X(t)
Y(t-1)
sensors
effectors
L
Aπ(t) SL
π(t-1)
X(t)
M =
{ Y(t) U
π(t) }
π(t)
Y(t)
C
A university meeting room in Second Life Seminars and Meetings Tutorials Skype-conferencing
A Curious, Virtual, Sentient Room
Meta-Sensors and Meta-Effectors
BLIP System provides an up-to-date list of current sensors and effectors and acts as an intermediary for communication
Agent does not communicate directly with sensors and effectors
Agent has a ‘sensor of sensors’ and an ‘effector of effectors’
The Curious Room Agent
Computational model of novelty used for curiosity
Table-based supervised learning using associations Learns accurately but Unable to generalise
Preliminary Evaluation
~6 repetitions by human controlled avatars required for learning
Can adapt to new devices
Can adapt simple behaviours to form more complex sequences
Limitations
Current prototype is proof-of-concept only, no significant empirical results yet
Issue of if/when/how to ‘forget’ behaviours Is an interface required for
manual editing or override of learned behaviours?