Advanced Neural Implantsand Control

Daryl R. KipkeAssociate ProfessorDepartment of BioengineeringArizona State UniversityTempe, AZ 85287kipke@asu.edu

Approved for Public Release, Distribution Unlimited: 01-S-1097

The Underlying Premise…

The ability to engineer reliable, high-capacity direct interfaces to the brain and then integrate these into a host of new technologies will cause the world of tomorrow to be much different than that of today.

However…

� There are some serious scientific barriers between where we stand today and where we can stand in the future. • How do we establish permanent and reliable interfaces to

selected areas of the central nervous system?

• How do we use these interfaces to directly and reliably communicate at high rates with the brain?

Applied Neural Implants and Control

Project Director Kipke (BME)

AdvisoryCommittee Raupp,Hoppensteadt,Farin

Visualization & Modeling Farin (CSE) Nelson (CSE) Razdan (CSE) Smith (Math)

Systems Science &Signal Processing He (BME) Hoppensteadt (Math &EE) Kipke (BME) Si (EE)

Neural & Tissue Engineering Kipke (BME) Massia (BME) Panitch (BME) Rousche (BME)

Tissue Culture & Analysis Capco (Bio) Massia (BME) Pauken (Bio)

Materials Synthesis& Bioactive Coatings Ehestraimi (BME) Massia (BME) Panitch (BME) Raupp (ChemE)

MEMS Shen (EE) Pivin (EE) Li (EE)

INFO

BIO

MICRO

Primary Goals of theBIO:INFO:MICRO Project

� Develop new neural implant technologies to establish reliable, high-capacity, and long-term information channels between the brain and external world.

� Develop real-time signal processors and system controllers to optimize information transmission between the brain and the external world.

SysSci VizMod

NeuEng MEMS

TisClt

Mat'lSyn

Systems-level Approach…

Feedback control signals

Subject Neural system

(global)

Controlled neural plasticity

local

Neural Implant

Adaptive Controller

External World

Objective 2: Optimize Adaptive Controller

Objective 1: Optimize neural interface

Topics

� Project overview

� Towards the Development of Next-Generation Neural Implants (BIO, MICRO, and INFO)

� Bioactive Coatings to Control the TissueResponses to Implanted Microdevices

� Modeling the Device-Tissue Interface

� Direct Cortical Control of an Actuator

� Neural Control of Auditory Perception

� Wrap-up

Focus on Next-GenerationNeural Implants

Feedback signals: local

Subject Neural system

(global)

Controlled neural plasticity

local

Neural Implant

Neural Controller

External World

host response

Info. Signals: electrical & chemical

Objective 2: Optimize Adaptive Controller

Objective 1: Optimize neural interface to achieve reliable, two-way, high-capacity information channels. …and “self-diagnostic”

Fundamental Problem of ImplantableMicroelectrode Arrays

� Brain often encapsulates the device with scar tissue � Normal brain movement may cause micro-motion at the tissue-

electrode interface � Proteins adsorb onto device surface � Useful neural recordings are eventually lost

Electrode 1

Electrode N

Implant Failure

Month 1Implant Month N

3rd-Generation Neural Implants

Technology Spectrum

1st-generation

Microwires

2nd-generation

Silicon arrays 3rd-generation Neural Implants

Desired Properties • Very high channel count

(<1000)

• Bioactive coatings

• Flexible

• Engineered surfaces

• Controlled biological response

• Integrated electronics

“Brain-centered” Design of Neural Implants

Initial conceptual designs

B

B

A

A

A A B B

through hole connecting channel

recording site

bioactive gel

Standard Perforated Probe

Simple Bioactive Probe

Differential Bioactive Probe

through holerecording site

bioactive gel

flexible polyimide substrate

bond pads

e.g. corticosteroid NGF e.g. GABA

cross-section (A-A) cross-section (B-B)

Polymer-substrate Neural Implants

• 2-D planar devices can be bent into 3-D structures • Increases insertion complexity

Holes to promote integration with neuropil

90 degree angles

Recordings From Polymer-substrateNeural Implants

Chan. 9

Chan. 10

One Day Post-op

Lost most unit activity after 7 days – Most likely due to failure to properly close dural opening.

Flexible Neural Implants PresentSurgical Challenges

� While the “micro-motion” hypothesis suggests that flexible neural implants should be more stable, the same flexibility presents significant new surgical challenges.

“Difficult” insertion “Easy” insertion

Rdr2, 9-00 Rdr3, 9-00

Using Dissolvable Coatings toStiffen the Neural Implant

� Dip-coat microdevice with polyethylene glycol (PEG) • Provides mechanical stiffening prior to implant • Quickly dissolves when in contact with tissue

First insertion of coated microdevice into Second insertion of coated microdevice

gelatin -- Device easily penetrates into gelatin – The device is too flexible to

material penetrate material because the PEG has dissolved.

Micromachined Surgical Devices

Vacuum nozzle

Flexible probe

Insertion aid

Vacuum Actuated Knife/Inserter

PEG

Silicon Knife/Inserter

Exploratory Functionality

Bioactive Component Storage Structures

Passive Surface Engineering

Active FET Devices, ChemFETs

Electrical Recording/Stimulating

Surfaces

Other Active Devices (Thermal, Magnetic, Strain, etc.)

Fluid Microchannels

Polymer Substrate

• Magnetic/thermal stimulation

• Drug delivery channels

• Active micro-manipulation of probes

Currently...

Internal Review

Feasibility Studies Insertion Aids

Mechanical Transfer Structures

Signal Processing Termination

Multiple Dimensions and Forms

Implant Coatings and Surface Modifications

Parylene-N,C Photo-crosslinked PolyimidesCl

Cl O O

O

n

C

C

C

C

smoothsmoothsmooth porousporousporous

N N

OO

Surface Plasma Treatments NH2 NH2 NH2 NH2

(NH3 - Amination)

Advanced Neuro-Device Interfaces

Passive

Chemical/Electronic NHNH22 NHNH22 NHNH22NHNH22

Amplification

ion beammetal

modified regionsite or interdigits

release layer polymer (PI/P-C)

or substrate

Active

Silicon FETs?

Topics

� Project overview � Towards the Development of 3rd-Generation

Neural Implants (BIO, MICRO, and INFO)

� Bioactive Coatings for Controlled Biological Response (BIO, MICRO, and INFO)

� Modeling the Device-Tissue Interface

� Direct Cortical Control of an Actuator

� Neural Control of Auditory Perception

� Wrap-up

Approach

Advanced biomaterials and micro-devices for long-term

implants (BIO, MICRO, INFO)

Cellular and biochemical response characterization

(BIO, MICRO)

Models and 3-D visualization of device-tissue dynamics

(BIO, INFO)

Engineer the neural implant surface in order to control both the material response and the host response.

Factors Limiting ChronicSoft Tissue Implants

� Inability to control cellular interactions at biomaterial-tissue interface

� Initial adsorption of biological proteins • Non-selective cellular adhesion

� Unavoidable “generic” foreign body reactions • Inflammation

• Fibrous capsule formation

Potential Solution

� Engineer surface for minimal protein adsorption and selective cell adhesion • Cell-resistant polymer coatings

• Synthetic: Polyethylene Glycol, Polyvinyl Alcohol

• Natural: Polysaccharides, Phospholipids

• Surface immobilization of biologically active molecules

• Mimic biochemical signals of extracellular matrix

• Cell binding domains for integrin receptors

Biomimetic Surface Modification

NH2 NH2 OH

O

HO

N

O O

OH

OH

O O

OH

OH

O

HO HOHO

O

OH

OH

OO

HO

N OHHO

O

HO

O

NTF NTF

Material Surface

Recombinant NGF Fusion Protein

Factor IIIa

Active or inactive plasmin­degradable substrate

Degraded plasmin­substrate

substrate

Human b-NGF

plasmin

Fibrin

Plasmin cleavage Human b-NGF

Fibrin

Bioactive Functionality

Methods 6-hour diffusion in rat cortex

Fluorescence Intensity Prof i le

250

NeuroTrace� DiI tissue-labeling paste, inverted fluorescent microscope with FITC/rhodamine filter cube

200

150

Pix

el

Va

lue

100

5 0

0

0 2 0 4 0 6 0 8 0 100 120 140 160

D i s t a n c e ( m i c r o n s )

Topics

� Project overview � Towards the Development of 3rd-Generation

� Bioactive Coatings to Control the Tissue Neural Implants (BIO, MICRO, and INFO)

Responses to Implanted Microdevices (BIO, MICRO, and INFO)

� Modeling the Device-Tissue Interface (BIO, MICRO, and INFO)

� Direct Cortical Control of a Motor Prosthesis � Neural Control of Auditory Perception � Wrap-up

The Device-Tissue Interface

Neural Interface: Micro-device, Neurons, Glia, Extracellular Space

The Goal is to Characterize, Predict, and Controlthe Device-Tissue Interface

Tissue State

(e.g., encapsulation,excitability)

Biophysical Model of the

Device-Tissue Interface

Device Function

(e.g., impedancespectrum)

• Integrate bioelectrical, histological and biochemical data • Optimize electrode specifications

Visualization of the Chronic Device-TissueInterface With Confocal Microscopy

A B

C D

In vivo Visualization of the ChronicDevice-Tissue Interface

Multi-Domain Continuum Model

( ) ( )

( )

/

/

/

At each "point" in space:

volume fraction

potential ,

conductivity tensor

membrane parameters

, , , etc.

e i

e i

e i

L

r

f

r t

G

C ga

F

r

r

r

r

• Tissue is two (or more) coupled volume-conducting media

• Electrode is boundary condition

r

r

Equations for a Multi-DomainContinuum Model

Volume conductor equations (conservation of current)

- fe��(Ge�Fe ) = +� Imemi + Iapp

i

- fi��(Gi�Fi ) = -Imemi i = index over intracellular domains

Membrane potential(s) and membrane current(s)

� ¶V �Vi = F i - F e Imemi = ai �

Ł Ci ¶t

i + Iioni �ł -1

F = potential (mV) a i = surface to volume ratio (cm ) Vi = membrane potential (mV)/e i 3 2Ge i = conductivity (mS/cm) Imemi

= membrane current (mA/cm ) Ci = membrane capacitance (mF/cm )/

3 2fe i = volume fraction Iapp = applied current (m A/cm ) Iioni = membrane current (mA/cm )/

Levels of Modeling

NumericalMultiple intracellular domains

Voltage-dependent conductances

ioni = � g ij � qijk (Vi - E j )

j k

¥¶qijk qijk -q Vijk ( i )= -¶t t ijk (Vi )

Complex electrode geometry

Tissue inhomogeneous and anisotropic

under construction

AnalyticalA single intracellular domain

Passive membrane conductance

I ion = g L (V EL )-

Simple electrode geometry

Tissue assumed homogenous and isotropic

much progress

I

Bi-domain Model for theMicrocapillary Bioreactor

Calculate profiles

F1 e i (x;w)/

in bioreactor

...and impedance...

Z(w) =

F1 e ( L;w) -F1

e (0;w) j1

100 Hz

1 1 / Write BCs and assume: ( , ) ( ; )i t i t

e i e i j j e x t xw w= � F = F

Z

w

( )

/

...and predict

as tissue parameters

, , , , ,

are experimentally

manipulated

e i e i L

Z

f G C g E

w

a

V

e F

i F L E

/ ew

/ L

Recap

� Focused & integrated effort • BioMEMS…Neural

Engineering…Materials… Computational Neuroscience…Cellular Biology…Visualization

� Why are we so excited?• We have the very real

potential of characterizing the biological responses to neural implants and then engineering new classes of microdevices to provide a permanent high-capacity interface to the brain

BIO

INFO

MICRO

Why the BIO, INFO, andMICRO Program?

� Wide-open Challenges • Characterizing and modeling the biological (cellular and

chemical) responses around a neural implant • Controlling the dynamic biological responses around a neural

implant. • Designing, fabricating, and using “advanced” neural implants

� Collaboration Possibilities • Additional functionalities for implantable microdevices of the

class that we are working on. • Exploring fundamentally new types of tissue-device interfaces. • Complementary studies of the neural interface (experimental

and analytical) • Confocal microscopy of the neural interface • Sharing technologies, procedures, insights, etc… • New emergent ideas…

Systems-level Analysis of AdvancedNeuroprosthetic Systems

Feedback control signals

Subject Neural system

(global)

Controlled neural plasticity

local

Neural Implant

Adaptive Controller

External World

Objective 2: Optimize Adaptive Controller

Objective 1: Optimize neural interface

Systems-level Approach for AdvancedNeuroprosthetic Systems

Subject Neural system

(global)

Controlled neural plasticity

local

Neural Implant

Adaptive Controller

External World

Feedback control signals

Objective 2: Develop Objective 1: Optimize neuraladaptive controller to interface

optimize systemperformance.

Advanced Neuroprosthetic Systems

High-Level Neural Computation

Sensory Transduction & Pre-processing Motor

Commands Movement

Perception, Decision, Detection

Sensory Integration

External World

Neuroprosthetic System

� Underlying System Principles •Two-way communication with targeted neural systems

•Harness neural plasticity to our advantage

•Appropriately balanced “wet-side” and “dry-side” computation

Approach

� Four Project Areas �Direct neural control of actuators

�Detection of novel sensory stimuli through monitoring neural activity

�Neural control of behavior

�Investigate signal transformations from ensembles of single neurons to local field potentials to EEG.

Topics

�

�

�

�

Project overview

Towards the Development of 3rd-Generation Neural Implants (BIO, MICRO, and INFO)

Bioactive Coatings to Control the Tissue Responses to Implanted Microdevices (BIO, MICRO, and INFO)

Modeling the Device-Tissue Interface (BIO, MICRO, and INFO)

� Direct Cortical Control of a Motor Prosthesis(BIO, MICRO, and INFO)

� Neural Control of Auditory Perception

� Wrap-up

Direct Cortical Control of Actuators

High-Level Neural Computation

Sensory Transduction & Pre-processing Motor

Commands Movement

Perception, Decision, Detection

Sensory Integration

External World

NeuroprostheticSystem

Goal: Control arm-related actuator

External Actuator Robotic Arm or Virtual Reality

Fundamental Questions

� What are “optimal” real-time signal processing strategies for precise 3-D control of external, arm-related actuators in the presence of sensory distractions and/or physical perturbations to the arm?

� To what extent can we use composite neural signals [neuronal (unit) recordings, local field potentials, and brain-surface recordings] for control signals?

� How do we take advantage of inherent or controlled neural plasticity in order to optimize system performance?

Experimental Preparation

• Train monkeys to perform tracking and/or reaching tasks.

• Record cortical responses with multichannel neural

implants.

• Measure arm movement in 3-D space.

Chronic Neural Recordings

� Multi-channel neural implants in motor and sensorimotor cortical areas.

� Eventually: Sub-dural electrodes for local potentials

-0.2 0 0.2 0.4 0.6

0

10

dsp009b

-0.2 0 0.2 0.4 0.6

0

20

40

dsp012a

-0.2 0 0.2 0.4 0.6

0

10

20dsp018a

-0.2 0 0.2 0.4 0.6

0

20

40

dsp024a

-0.2 0 0.2 0.4 0.6

0

20

40dsp025a

-0.2 0 0.2 0.4 0.6Time (sec)

0

10

dsp030a

-0.2 0 0.2 0.4 0.6

0

100

dsp034a

-0.2 0 0.2 0.4 0.6

050

100150

dsp037a

-0.2 0 0.2 0.4 0.6

05

10

15dsp040a

-0.2 0 0.2 0.4 0.6

0

10

20

dsp042a

-0.2 0 0.2 0.4 0.6

0

20

dsp042b

-0.2 0 0.2 0.4 0.6Time (sec)

010

20

30dsp045a

-0.2 0 0.2 0.4 0.6

0

20

40

dsp046a

-0.2 0 0.2 0.4 0.6

0

5

1015

dsp051a

-0.2 0 0.2 0.4 0.6

0

20

40

dsp057a

-0.2 0 0.2 0.4 0.6

0

40

80

dsp058a

Perievent Histograms Target 1, reference = C_rel, bin = 20 ms

NeuralRecordingSystem

OfflineAnalysis

Real-timeSignalProcessing

ActuatorControl

Extracellular recordings

Direct Cortical Control of Movement

Green ball: Target Yellow ball: Actual hand position, or hand position estimated from cortical responses

m0602pa

Topics

� �

�

�

�

Project overview Towards the Development of 3rd-Generation Neural Implants (BIO, MICRO, and INFO) Bioactive Coatings to Control the Tissue Responses to Implanted Microdevices (BIO, MICRO, and INFO)

Modeling the Device-Tissue Interface (BIO, MICRO, and INFO)

Direct Cortical Control of a Motor Prosthesis (BIO, MICRO, and INFO)

� Neural Control of Auditory Perception(BIO, MICRO, and INFO)

� Wrap-up

Neural Control of Auditory Perception

High-Level Neural Computation

Sensory Transduction & Pre-processing Motor

Commands Movement

Perception, Decision, Detection

Sensory Integration

External World

NeuroprostheticSystem

Goal: Control auditory perception

Fundamental Questions

� To what extent can we control auditory-mediated behavior using intra-cortical microstimulation (ICMS) through the neural interface?

Transmitter Channel Receiver Source Signal

Received Signal

Stimulator Neural Interface

Auditory Cortex

� What are the information transmission characteristics of the multichannel neural implant in high-level cortical areas using ICMS? � Channel capacity (bits per second) � Channel reliability � Channel resolution

� How can we optimize information transmission � Implant designs, Neural implant locations, Signal encoding strategies,

Controlled neural plasticity

Chronic Neural Recordings

� Multi-channel neural implants in primary auditory cortex

Extracellular recordings in auditory cortex

Estimation of

Neural Recording System

Offline Analysis

Neuronal ResponseProperties

Algorithm Selection

Signal Encoder

SoundsElectrical Stimulation to Aud. Ctx.

Behavioral performance to both sounds and cortical electrical stimulation

Auditory Behavior

• Lever-press sound or ICMS discrimination task

• Center paddle hit starts trial, 2-tone pair presented

• Reward obtained by signaling the correct stimulus sequence

centerleft right

rat

Frequencyresponse areas

Frequency Selectivity in Auditory Cortex

1 2 5 10 20 30

40

60

80dsp024b

28.

56.

1 2 5 10 20 30

40

60

80dsp018d

11.

22.

1 2 5 10 20 30

40

60

80dsp020a

10.

20.

1 2 5 10 20 30

40

60

80dsp024a

10.

20.

1 2 5 10 20 30

40

60

80dsp012a

21.

42.

1 2 5 10 20 30

40

60

80dsp018b

10.5

21.

1 2 5 10 20 30

40

60

80dsp018c

22.

44.

1 2 5 10 20 30

40

60

80dsp002a

3.

6.

1 2 5 10 20 30

40

60

80dsp002b

5.5

11.

1 2 5 10 20 30

40

60

80dsp010b

12.

24.

Freq.

SoundLevel

Signal Encoding Algorithm:Frequency Selectivity

ICMS pattern is based solely on frequency selectivity of neurons recorded on an electrode

dB

80

60

40

u5b 8

6

Spik

es

4

2

01 5 10 30

kHz

u32a

0

2

4

6

8

kHz

1 5 10 30

Spik

es

80

60dB

40

Behavioral Performance

Ricms6

Rat Behavioral Performance

RICMS 6

100

09/06

/00

09/16

/00

09/26

/00

10/06

/00

10/16

/00

10/26

/00

Training day Implanted

90

80

Per

cen

t Co

rrec

t

70

60

50

40

30

20

10

0

Cortical Electrodes

D

Expected Results to ICMS Stimuli

Begin ICMS

100

% D% due to ICMS

Trial #

Auditory trial =

ICMS Algorithm1 =

ICMS Algorithm2 =

Behavioral Curve

RICMS 6 10/25 (Only Session)

100

80

Per

cen

tag

e

audPercent, icmsPercent,

60

40

20

0

0 100 200

Trial

Alternative Signal Encoding Algorithm:Cortical Activation Pattern

For a given electrode, the unit firing pattern is used as a template for ICMS delivery

Auditory Stimulus

Sound on

Response Raster

Matching ICMS ‘pattern’

***Procedure is simultaneously duplicated on each active electrode

Recap

� Focused & integrated effort • Neural Engineering…Signal

Processing…Systems Neurophysiology…Visualization

� Why are we so excited? • We have the very real

potential of developing new classes of neuroprosthetic systems to explore our ability to interact directly with the brain.

BIO

INFO

MICRO

BIO, INFO, and MICRO…

� Wide-open Challenges • Appropriate mathematical constructs for describing neural

encoding and decoding.

• Advanced data visualization techniques for understanding this new class of neural data.

• Understanding signal transformations as a function of the spatial and temporal scale of the neural data.

� Collaboration Possibilities • Exploring new signal encoding and decoding strategies for

particular neuroprosthetic applications.

• Sharing technologies, procedures, insights, etc…

• New emergent ideas…

Topics

�

�

�

�

�

�

Project overview

Towards the Development of 3rd-Generation Neural Implants (BIO, MICRO, and INFO)

Bioactive Coatings to Control the Tissue Responses to Implanted Microdevices (BIO, MICRO, and INFO)

Modeling the Device-Tissue Interface (BIO, MICRO, and INFO)

Direct Cortical Control of a Motor Prosthesis (BIO, MICRO, and INFO)

Neural Control of Auditory Perception(BIO, MICRO, and INFO)

� Wrap-up

Project Challenges

� Scientific • Overcoming engineering and scientific hurdles.

• Identifying and fostering strategic alliances with appropriate external groups.

• Crossing disciplines

� Management • Strategic planning

• Resource allocation

• Open and effective communication among the diverse project team

• Team-building: Maintaining enthusiasm, energy, and focus after the initial “honeymoon” period

“Insanely IntenseInterdisciplinary” Research

“pieces of a puzzle” “easy synergism”

BIO

INFO MICRO

BIOMICRO

INFO

Breakthrough Science

•Hard work •Open minds •Honesty •Top-notch research

--

What Does the Future Hold?

“Perhaps within 25 years there will be some new ways to put information directly into our brains. With the implant technology that will be available by about 2025, doctors will be able to put something like a chip in your brain to prevent a stroke, stop a blood clot, detect an aneurysm, help your memory or treat a mental condition. You may be able to stream (digital) information through your eyes to the brain. New drugs may enhance your memory and fire up your neurons.” Dr. Arthur Caplan,

Director of the Center of Bioethics, University of Pennsylvania Arizona Republic, Dec 27, 1998.

Acknowledgments

� ASU Colleagues • 13 co-PI’s, 5 research faculty, numerous graduate

and undergraduate students.

� Arizona State University administration • Seed funding from Department, College, and

University

• Significant cost-share on this project

� DARPA Program Managers • Eric Eisenstadt, Abe Lee, and Gary Strong