towards real-time distributed signal modeling for brain ... · 1 towards real-time distributed...
TRANSCRIPT
http://nrg.mbi.ufl.edu1
Towards Real-Time Distributed Signal Modeling for Brain-Machine Interfaces
Justin C. Sanchez, Ph.D.
Neuroprosthetics Research Group (NRG)http://nrg.mbi.ufl.edu
On behalf of the DDDBMI PI team:José Fortes, Renato Figueiredo, Linda Hermer-
Vasquez, and José Principe
http://nrg.mbi.ufl.edu2
Enabling Neurotechnologies for Overcoming Neurological Disorders
Develop direct neural interfaces to bypass injury. Communicate and control (closed-loop, real-time) via the interface.Spinal Cord InjuryMovement DisabilitiesStroke
Leuthardt
http://nrg.mbi.ufl.edu
Goals of this Project
ImmediateA test bed for real-time, closed-loop BMI modeling and experimentation Advance behavioral brain modeling, BMI experimentation, algorithms and Grid-computing.
Long-TermCyberworkstation for real-time neurophysiologicalexperiments.A general purpose platform for modeling and experimentation in neurophysiology. Impact the speed and complexity of new ideas that can be tested in the neurophysiology laboratory
http://nrg.mbi.ufl.edu
Integrating the Multidisciplinary Team
Here we present the critical architecture and infrastructure to support theory of brain information processing and motor control function.
http://nrg.mbi.ufl.edu
BMI Portal – Provides the Bridge
Reservation of resourcesFor online experiments
Access to data setsFor replay and analysis of
experiments
Specification of modelsFor use in either offline or
online experiments
Access to computational tools
For analysis, simulation, visualization ….
http://nrg.mbi.ufl.edu
Middleware Architecture
http://nrg.mbi.ufl.edu
BMI Workstation Components
Data ExchangesRat & Pentusa
PentusaData.exe GlobalBMI LocalBMI
Bin numbers, masks, thresholds
InputDatInputDat
OutputDat
Robot Command
Robotic Arm
NRG ACIS
http://nrg.mbi.ufl.edu
Data Flow and Scheduling
http://nrg.mbi.ufl.edu
Multiple paired models and the responsibility predictor for the DDDAS based BMI
http://nrg.mbi.ufl.edu
General RequirementsNumber of pairs of “internal” models
10s – 100s for simple tasks (e.g. press lever)1000s (?) for complex tasks
Types of “internal” modelsLinear (filters): Wiener, NLMS, PVA, …Nonlinear (neural nets): TDNN, RMLP, RNN, NMCLMState-based: Kalman filters, Bayesian classifiers, HMMs, RLBMI
Complexity of modelsO(n), O(n2), O(mn2), O(n3), …for n neurons, m models
http://nrg.mbi.ufl.edu
One Realization of the Computational Structure
Online – real-time (100ms - hard deadline)Offline – recreation of experiments from data in storage
module module module…
data
…
training/gating
…
+10 – 100 msHard (neural
sampling)
10 – 500 msSoft (model update)
Reinforcement Learning
http://nrg.mbi.ufl.edu
Experimental Paradigm
-3 -2 -1 0 1 2 3
01
23
0
1
2
3
IncorrectTarget
CorrectTarget
StartingPosition
Match LEDs
Grid-space
Match LEDs
Rat’s Perspective
Water Reward
Map workspace to grid
Rat
Robot Arm
Left Lever Right Lever
http://nrg.mbi.ufl.edu
Feasibility of Architecture –Offline Performance Evaluation
7.3%34.3%43.8%MLP**3
11%23.1%43.8%MLP*2
Xx31.2%SLP1
10.8%68.1%81.3%MLP**3
10%61.9%81.3%MLP*2
xx93.7%SLP1
VarianceMeanMaxStructureDimension
*H = 2 **H = 3SLP – Single Layer Perceptron
MLP – Multilayer Perceptron
Surrogate
http://nrg.mbi.ufl.edu
Real-Time Experimentation across Campus
ACISlab
NRGlab
http://nrg.mbi.ufl.edu
Closed-loop BMI Timing Result
No deadline missedTotal closed loop time allows more modules to be addedVirtualization can improve resource utilization
17.5
62
3.59
4471 6.50
2927
4.27
0123
0.04
8975
1
24.1
83
2.40
0925
0
5
10
15
20
25
30
Computation (µs) Training (ms) Acquisition (ms) Transfer (ms) Robot (ms)
Physica ResourceVirtual Resource
http://nrg.mbi.ufl.edu
Real-Time, Closed-Loop Robot under Brain Control using DDDBMI Architecture
Non-functional levers
Functional levers
Robot workspace in rat visual field of view.
BLUE – Robot
GREEN - Lever
Top-view of the rat behavioral cage.
http://nrg.mbi.ufl.edu
Milestones MetNeural recordings time synchronized with behavior from multiple arrays. Assessment of (open loop and closed loop) cortical (M1) contribution to the lever pressing task. Statistical comparison of training and testing MSE for the proposed algorithms (80-90% accuracy). Benchmark of virtual application performance. All met 100ms harddeadline. Implemented and tested gating function training (Reinforcement Learning) Evaluated parameters for symbiotic training using VM cluster in real-time closed loop operation Implemented spike train learning for the forward modelInitiated study of adaptive virtual application rescheduling, VMreservation middleware with animal dataSpecified bottlenecks in computing architecture for real-time experiments.
http://nrg.mbi.ufl.edu
Student Support through the DDDASBMI
18
Prapaporn Rattanatamrong -ECE
Jack DiGiovanna -BME
BabakMahmoudi -BME
ShalomDarmanjian -ECE
Ming Zhao -ECE
This work is supported by NSF project No. CNS-0540304