brain machine interfaces for motor control: building adaptive
TRANSCRIPT
http://nrg.mbi.ufl.eduhttp://nrg.mbi.ufl.edu
Symbiotic Brain-Machine Interfaces
Justin C. Sanchez, Ph.D.
Assistant ProfessorNeuroprosthetics Research Group (NRG)
University of Floridahttp://nrg.mbi.ufl.edu
http://nrg.mbi.ufl.edu
Enabling Neurotechnologies for Overcoming Paralysis
Develop direct neural interfaces to bypass injury. Communicate and control (closed-loop, real-time) directly via the interface.
Leuthardt
http://nrg.mbi.ufl.edu
Vision for BMI in Daily Life
Lebedev
http://nrg.mbi.ufl.edu
What are the Building Blocks?
Signal SensingAmplification
Pre-Processing
Telemetry
Interpret Neural Activity
Control
SchemeFeedback
Closed Loop BMI
Provide neurophysiologic basis and engineering theory for a fully implantable neuralInterface for restoring communication and control
http://nrg.mbi.ufl.edu
BMI lessons learnedRelationship between user
and BMI is inherently lopsided. Users are intelligent and can use dynamic brain organization and specialization while BMIs are passive devices that enact commands
I/O models have difficulty contending with new environments without retraining
Laboratory BMIs need to be better prepared for ADL
http://nrg.mbi.ufl.edu
Translating Thoughts into Action: The Neural Code
StimulusNeuralSystem Neural Response
Stimulus Neural Response
Coding Given To determine
Decoding To determine Given
http://nrg.mbi.ufl.edu
Vision for Next Generation Brain-Machine Interaction
Intelligent behavior arises from the actions of an individual seeking to maximize received reward in a complex and changing world.
Perception-Action Cycle: Adaptive, continuous process of using sensory information to guide a series of goal-directed actions.
http://nrg.mbi.ufl.edu
Co-Adaptive BMIs using Reinforcement Learning
http://nrg.mbi.ufl.edu
Prerequesites for Symbiosis
http://nrg.mbi.ufl.edu
Co-Adaptive BMI involves TWO intelligent agents involved in a continuous dialogue!!!
ROBOT
action
s
rew
ard
s
bra
in s
tate
s
RAT’S BRAIN
environment
RAT’S BRAIN
COMPUTER AGENT
http://nrg.mbi.ufl.edu
Decoding using Reinforcement Learning
Rather than knowledge of the kinematic hand trajectory only a performance score is supplied. The score could represent reward or penalty, but does not directly provide information about how to correct for the error.
Reward based learning - try to choose strategy to maximize rewards.
RL originated from optimal control theory in Markov Decision Processes.
€
Rt = γ n−t +1rn
n= t +1
∞
∑
€
Q st ,at( )*
= E Rt | st ,at{ }
http://nrg.mbi.ufl.edu
Experimental Co-Adaptive BMI Paradigm
-3 -2 -1 0 1 2 3
0
1
2
3
0
1
2
3
IncorrectTarget
CorrectTarget
StartingPosition
Match LEDs
Grid-space
Match LEDs
Rat’s Perspective
Water Reward
Map workspace to grid
Rat
Robot Arm
Left Lever Right Lever
27 discrete actions 26 movements
1 stationary
http://nrg.mbi.ufl.edu
Agent - Value function estimation
€
Qk (v s t ) = tanh si,t
i
∑ wij
⎛
⎝ ⎜ ⎜
⎞
⎠ ⎟ ⎟
⎛
⎝ ⎜ ⎜
⎞
⎠ ⎟ ⎟w jk
j
∑
€
δt = rt +1 +γQ(st +1,at +1) −Q(st ,at )
http://nrg.mbi.ufl.edu
Evidence for Symbiosis
Valuation Change in Computer Agent
Brain Reorganization
Overall Performance
http://nrg.mbi.ufl.edu
Key Concepts for the Future
Fully implantable interfaces are only half of the story.
Sharing of goals enables brain-computer dialogue and symbiosis
Need for intelligent decoders that assist and co-adapt with the user.
http://nrg.mbi.ufl.edu
History of Man-Machine Interaction
“Implanting tiny machines into the nerves of the heart would make us less human”
Today, over half a million pacemakers are implanted annually!
We are at the frontier for integrating machines with the nervous system to restore and enhance function.
Nicolelis
http://nrg.mbi.ufl.edu
Tremendous team effort!
Jack DiGiovanna - BME
Babak Mahmoudi - BME
This work is supported by NSF project No. CNS-0540304
Jose Principe - ECE
Jose Fortes - ECE
http://nrg.mbi.ufl.edu
Please visit the lab website for publications and additional information.
Neuroprosthetics Research Group http://nrg.mbi.ufl.edu