krishna jharjaria tj strzelecki rick schuman matt waldersen
TRANSCRIPT
Team 2: Mind Readers
Krishna Jharjaria TJ Strzelecki Rick Schuman Matt Waldersen
Project Overview
The Mind Reader is a mobile brain-computer interface. Computer applications will be presented to the user
through commercially available video glasses. An EOG and commercially available EEG will be
mounted inside of a common enclosure and will enable the user to navigate and select various applications.
A dsPIC microcontroller will be used to acquire the EOG and EEG signals, the EEG signals will be analyzed by an FPGA and a BeagleBoard XM will control the virtual reality environment as well as execute all of the computer applications
Project-Specific Success Criteria
1) An ability to encode/decode data packets from a NeuroSky EEG.
2) An ability for a user to select applications based on signals from a NeuroSky EEG.
3) An ability for a user to navigate between different applications on a display using EOG signals.
4) An ability for the system to interactively train the user to effectively operate the device.
5) An ability to display a live video stream from an external camera module, and integrate applications into the video system.
Block Diagram
Component Selection RationaleMicrocontroller
Considerations Signal Processing abilities Digital Communication pins Optimized for C compiler Processing speed Resources and reference material
DSPIC33EP512MU810 Extensive DSP Library with built in FFT function 4-UART; 4-SPI; 2-I2C Optimized for C compiler ~53K of RAM Large online community
EOG Op-Amp Selection Rationale (TLC2272/4 & TLV2211 )
Low Noise Dual Polarity Model Available in PSPICE Multiple Op-Amp Models Available in DIP and SMD
configurations With exception of the TLV2211
High Accuracy
EOG Op-Amp Selection Criteria
ADC Selection Rationale (ADS1210)
High Resolution 19.5 Effective Resolution (Bits RMS)
Differential input and 2.5V reference output eliminates need for virtual ground
Integrated Programmable Gain Amplifier
Integrated Digital Filter Available in both DIP and SMD
packages
Considerations Size (Logic Blocks) Number of I/O Pins Built in Functionality Power Consumption
DLP-HS-FPGA3 USB - FPGA Module 25,344 Logic Blocks (11,264 slices) 63 Available user I/O channels 32 Dedicated Multipliers, libraries for floating point arithmetic FPGA module contains: regulators, clock generator, SDRAM,
USB programmer, SPI Flash, compatible with Xilinx ISE WebPack
FPGA operates on 3.3V, sinks/sources approximately 24mA per I/O pin
Component Selection RationaleFPGA Module
BeagleBoard-xM 1.0 GHz ARM Cortex-A8 Processor 512 MB RAM 8 GB microSD HDD ▪ libraries and main program
4 USB 2.0 Slots ▪ Mouse/Keyboard dev, Webcam
Expansion Header (Data Transfer)▪ o SPI▪ o UART▪ o I2C▪ o GPIO
Component Selection RationaleBeagleBoard xM
Packaging Design
Modular design Maintains separation of digital circuitry
and sensitive analog EOG circuitry Enables the EOG circuitry to be placed
as close possible to signal electrodes Video glasses will be used in order to
ensure that the video display is directly in front of the field of view
Light Weight Single Board Computer
Packaging Design
BeagleBoard-xM Size: 3.35” x 3.45” Weight: < 1 lb.
Logitech C310 HD Cam 1280x720p frame capture < 8 oz.
Vuzix Wrap 920 Twin 640x480p Display < 3 oz.
Main Circuit BoardEOG inputs
Power SupplyMicrocontroller
Interface
FPGA and Beagle Board
Power Supply
11.1V rechargeable Lithium ion Battery
5V convertorFor FPGA and Beagle Board
3.3 V convertorFor Microcontroller
2 Switch Mode regulators
EOG inputs
Connected to EOGvia SPI
Connected to EOG via SPI
Logic Level convertorFrom 5V to 3.3V
Registered Jack (RJ11)
Micro, in-system-programmer, EEG Dongle, Oscillator
Oscillator
In System Programmer
PCB mountReset
Through holeReset Connections
Switch debouncer
Pin 13 MCRL
Pin 15 PGECPin 16 PGED
EEG input via UART
UNIVERSAL ASYNCHRONOUS RECEIVER TRANSMITTER (UART)
Baud rate = 115200 bps Specified by the NeuroSky MindWave
EEG signals FFT
EEG frequency range 4-7hz
Array size 512
Output of the FFT resides in RAM
Inbuilt FFT API: FFTComplexIP() BitReverseComplex() SquareMagnitudeCplx()
4-7hz
FPGA Module, Logic Level Converter, Beagle Board
Beagle Board
FPGA Module
Logic Level ConvertorFrom 3.3V to 5V
Select button
Switch debouncer
4 bit EOG outputUp, down, left, right
2x5 bit EOG input
4 I/O for EOG3 SPI pins for EEG
PCB LayoutDesign Considerations
Main Board Multiple Voltages needed Power Traces at least 40 mil, signal traces no smaller than
10 mil Isolate oscillator Physical location of connectors Decoupling Capacitors
EOG Board Sensitive analog circuitry Distance between electrodes and filtering/amplification Physical size of EOG PCB
PCB Layout
PCB Layout – Main BoardFGPA Module
PCB Layout – Main BoardFGPA Module
PCB Layout – Main BoarddsPIC Micro
PCB Layout – Main BoarddsPIC Micro
PCB Layout – Main BoardPower Supply
PCB Layout – Main BoardPower Supply
PCB Layout – Main BoardEOG/SBC Connection & LLT
PCB Layout – Main BoardEOG/SBC Connection & LLT
EOG Theory of Operation
Electrooculography measures the signal given off of corneo-retinal potentials in the eyes
An EOG circuit consists of an instrumentation amplifier, amplifiers, low pass filtering and high pass filtering
A high pass filter is used to eliminate a naturally occurring DC drift
A low pass filter is used to remove all other external noise
Instrumentation AmplifierInstrumentation
Amplifier High Input
Impedance Includes both a
high pass filter to eliminate DC drift and a second order low pass filter
Instrumentation Amplifier
Sallen-Key 2nd Order LPF
Sallen-Key 2nd-Order Low Pass Filter Eliminates noise from external sources
Sallen-Key 2nd Order LPF
First Order LPF - AmplifierFirst Order Low Pass Filter – Amplifier Circuit Eliminates noise as well as amplifies the signal
First Order LPF - Amplifier
Linear Regulator
Voltage Inverter
Analog-to-Digital Converter
Reference Ground & Input Jack
EOG Signal Conditioning
Voltage (V)
Time (s)
EOG Signal Conditioning
Voltage (V)
Time (s)
Normalizing EOG Signal
∆Voltage
Sample Number
Software Design/Development StatusArtificial Neural Network
What is an Artificial Neural Network? Simplified mathematical model of the human brain Utilizes a network of “neurons” to model relationships
between inputs and outputs How does a ANN work?
ANN implement non-linear functions in each neuron This function sums weighted input values and passes them
through a non-linear function, usually a Sigmoid function This output then propagates to further network layers for
the process to be repeated Why use an FPGA implementation
Highly parallel algorithm, with transistor like output Need for quick, complex calculations
Software Design/Development StatusArtificial Neural Network
Software Design/Development StatusArtificial Neural Network
Basics of A Neuron Each neuron contains a number of synapses, one for each
previous layer neuron connected A synapses will take inputs and multiply it by a predetermined
specific weight The weighted values will then be accumulated and funneled
into a nonlinear activation function (Sigmoid Function)
Hardware Design Considerations Need multiple multipliers, one for each synapses (38
multipliers)▪ Need to condense the size of a neuron▪ Using control logic and multiple clock cycles we can lower the
required multipliers to a single multiplier per neuron (12 multipliers) Implementation of non-linear Sigmoid Function▪ Look-up Table
Artificial Neural NetworkPreliminary Block Diagram
Software Design/Development StatusArtificial Neural Network
Where we proceed from here Training and testing of preliminary ANN using
Matlab ANN toolbox▪ This will allow the weighted values to be hardcoded into
the FPGA Design and testing of single neuron in VHDL Development of a proven working topology
Software Design/Development StatusHeads-Up Display
Tentative HUD Features
•Webcam Live Feed
•EEG Signal Data display
•Home Screen• 3 x 3 application layout• Concentration to select app
•Programs• Minimal key input• Closed apps return to Home
Software DesignSingleboard Program Flow Diagram
Project Completion Timeline
October 15-19 PCB Proof of Parts
October 22-26 Simple Software Suite Sample EOG circuit ADC microcontroller
communication October 29 – Nov 2
Micro to Single-board communication
EOG ANN finished in MATLAB
Nov 5 – Nov 9 Live Feed Visual Software
Suite EEG ANN prototyped in
MATLAB PCB assembled
Nov 12 – 16 EOG ANN finished on FPGA
Nov 19 – 23 Software Test Application User Training Application
Nov 26 – Nov 30 Troubleshoot
Dec 3 – Dec 7 Project Complete
Questions?
Reference (DSPIC33EP)
Reference (PIC Programmer)
Reference (RJ-12 Connection)
Reference (Oscillator/Reset)
Reference (DC Voltage Inv)
Reference (ADS1210 ATD)
Reference (TLC2272)
Reference (TLC2272)
Reference (TLV2211)
Reference(DLP FPGA Module)
Reference (MAX764-DC Inverter)