blue eyes technologies
TRANSCRIPT
BlueEyesHumanOperatorMonitoringSystem
Blue EyesTechnologies
•What isBlue Eyes ?
•Motivation
•Physiological foundations
•Systemdesign
•System implementation
•Futureimprovements
•Conclusion
BlueEyesHumanOperatorMonitoringSystem What is BlueEyes ?
• The complex solution for human-operator monitoring:– conscious brain involvement monitoring– triggering user-defined actions– data recording
What is BlueEyes not?• Doesn’t predict nor interfere with
operator’s thoughts• Cannot force directly the operator to work
•What isBlue Eyes ?
•Motivation
•Physiological foundations
•Systemdesign
•System implementation
•Futureimprovements
•Conclusion
BlueEyesHumanOperatorMonitoringSystem Who is BlueEyes for ?
• Generic control rooms– Power station– Flight control centers– Operating theatres – anesthesiologists
• Common application– A simpler system version for drivers
•What isBlue Eyes ?
•Motivation
•Physiological foundations
•Systemdesign
•System implementation
•Futureimprovements
•Conclusion
BlueEyesHumanOperatorMonitoringSystem Motivation
• Human error - a frequent reason of catastrophes and ecological disasters
• Lack of complex solution exploiting wireless link technology, database systems and AI algorithms
– temporal indisposition– weariness – oversight
• Monitoring of consciousbrain involvement
• Capable sensors available
•What isBlue Eyes ?
•Motivation
•Physiological foundations
•Systemdesign
•System implementation
•Futureimprovements
•Conclusion
BlueEyesHumanOperatorMonitoringSystem Wires, wires, wires...
A serious limitation of:
– Mobility– Comfort– Safety
•What isBlue Eyes ?
•Motivation
•Physiological foundations
•Systemdesign
•System implementation
•Futureimprovements
•Conclusion
BlueEyesHumanOperatorMonitoringSystem Personal Area Network !
A wireless Personal AreaNetwork
Solution:
•What isBlue Eyes ?
•Motivation
•Physiological foundations
•Systemdesign
•System implementation
•Futureimprovements
•Conclusion
BlueEyesHumanOperatorMonitoringSystem BlueEyes - benefits
• Prevention from dangerous incidents• Minimization of
– ecological consequences– financial loss– a threat to a human life
• The reconstructionof the course ofoperator’s work
•What isBlue Eyes ?
•Motivation
•Physiological foundations
•Systemdesign
•System implementation
•Futureimprovements
•Conclusion
BlueEyesHumanOperatorMonitoringSystem Physiological foundations
Preconscious brain
• manages the knowledge
• logical reasoning• makes decisions
• illogical• “automatic”
Conscious brain
Human brain
•What isBlue Eyes ?
•Motivation
•Physiological foundations
•Systemdesign
•System implementation
•Futureimprovements
•Conclusion
BlueEyesHumanOperatorMonitoringSystem Conscious brain - operation modes
Exploratory mode Planning mode
•What isBlue Eyes ?
•Motivation
•Physiological foundations
•Systemdesign
•System implementation
•Futureimprovements
•Conclusion
BlueEyesHumanOperatorMonitoringSystem Eye movements
Saccade - a rapid eye jump to a new location assigned by the conscious attention process.
Saccades presence ⇔ Exploratory mode
• Central vision andthe conscious brain
• Retina - central vision (macula)•What isBlue Eyes ?
•Motivation
•Physiological foundations
•Systemdesign
•System implementation
•Futureimprovements
•Conclusion
BlueEyesHumanOperatorMonitoringSystem The eye movement sensor
• Eye position measuring - direct infrared oculography
• Two axialaccelerometer
• Ambient lightsensor
Jazz Multisensor
•What isBlue Eyes ?
•Motivation
•Physiological foundations
•Systemdesign
•System implementation
•Futureimprovements
•Conclusion
BlueEyesHumanOperatorMonitoringSystem
• A reliable real-time buffering, processing and recording
• A clear visualization interface• Mechanisms for creating custom data
analyzers• To ensure a possibility of distributing the
processing among a number of computers
Design objectives
• A mobile data acquisition device
•Objectives
•System overview
•Motivation
•What isBlue Eyes ?
•Physiological foundations
•Systemdesign
•System implementation
•Futureimprovements
•Conclusion
BlueEyesHumanOperatorMonitoringSystem System overview
Central System UnitData Acquisition Unit
Atmel 89C52microcontroller
JazzMultisensor
ConnectionManager
Data Analysis
VisualizationModuleBluetooth Bluetooth
Bluetooth technology provides means for creating a Personal Area Network linking the operators and the central system.
DataLogger
•Objectives
•System overview
•Motivation
•What isBlue Eyes ?
•Physiological foundations
•Systemdesign
•System implementation
•Futureimprovements
•Conclusion
BlueEyesHumanOperatorMonitoringSystem Implementation - DAU
•DAU
•CSU
•Motivation
•What isBlue Eyes ?
•Physiological foundations
•Systemdesign
•System implementation
•Futureimprovements
•Conclusion
BlueEyesHumanOperatorMonitoringSystem Implementation - DAU
• Bluetooth module inside• Jazz sensor
interface• Microphone and
earphone sockets• 5-key keyboard• Two lines, 8
characters LCD• Powered from
6 AA batteries
•DAU
•CSU
•Motivation
•What isBlue Eyes ?
•Physiological foundations
•Systemdesign
•System implementation
•Futureimprovements
•Conclusion
BlueEyesHumanOperatorMonitoringSystem
BlueEyesData Acquisition Unit
ID card
DAU - components
MC 145483PCM codec
ID cardinterface
Simplekeyboard
Jazz Multisensor
Beeper
LCD display
LED indicators
Atmel 89C52microcontroller
UART MAX232
UAR
T
Bluetoothmodule
microphone
earphone
•DAU
•CSU
•Motivation
•What isBlue Eyes ?
•Physiological foundations
•Systemdesign
•System implementation
•Futureimprovements
•Conclusion
BlueEyesHumanOperatorMonitoringSystem DAU - features
• Lightweight• Runs on batteries - low
power consumption• Easy to use - does not
disturb the operator working
• ID cards for operator authorization
• Voice transmission using hardware PCM codec
Data Acquisition Unit
Atmel 89C52microcontroller
JazzMultisensor
Bluetooth
•DAU
•CSU
•Motivation
•What isBlue Eyes ?
•Physiological foundations
•Systemdesign
•System implementation
•Futureimprovements
•Conclusion
BlueEyesHumanOperatorMonitoringSystem Implementation - CSU
• Custom-builtBluetoothcommunicationroutines
• Intel USB camera• ID card programmer• Developed using Visual C++ 6.0 • Deployment environment:
Windows 2000,MS SQL Server
•DAU
•CSU
• Standard MDI userinterface
•Motivation
•What isBlue Eyes ?
•Physiological foundations
•Systemdesign
•System implementation
•Futureimprovements
•Conclusion
BlueEyesHumanOperatorMonitoringSystem CSU - saccade detector
• Saccade detector and conscious brain involvement level calculation
Eye movement velocity
-250-200-150-100
-500
50100150200250
0 8000 16000 24000 32000Time [ms]
Ve
locit
y
saccadeRaw eyemovement data
Differentiation
Conscious brain involvement level
Visual attention level
0
10
20
30
40
50
60
0 8000 16000 24000 32000
Time [ms]
Att
en
tio
n le
vel
(L
va
)
•DAU
•CSU
•Motivation
•What isBlue Eyes ?
•Physiological foundations
•Systemdesign
•System implementation
•Futureimprovements
•Conclusion
BlueEyesHumanOperatorMonitoringSystem CSU - Visualization Module
•DAU
•CSU
• Raw and processed data visualization using:
• audio and videodata playback
– VU-meters– pie-charts– time series
•Motivation
•What isBlue Eyes ?
•Physiological foundations
•Systemdesign
•System implementation
•Futureimprovements
•Conclusion
BlueEyesHumanOperatorMonitoringSystem
Bluetooth
Bluetooth
..
.
..
.
CSU - Connection Manager
BluetoothConnection
Manager
OperatorData
Manager
Alerts
Separateddata streams
...
...
OperatorManager
Alert buffer
Raw DataBuffer
Alerts
OperatorManager
Alert buffer
Raw DataBuffer
•DAU
•CSU
•Motivation
•What isBlue Eyes ?
•Physiological foundations
•Systemdesign
•System implementation
•Futureimprovements
•Conclusion
BlueEyesHumanOperatorMonitoringSystem CSU - features
Central System Unit
ConnectionManager
Data Analysis
VisualizationModuleBluetooth
• Connection management
• Data processing• Visualization• Data recording
• Accessverification
• System maintenance
ConnectionManager
•DAU
•CSU
DataLogger
•Motivation
•What isBlue Eyes ?
•Physiological foundations
•Systemdesign
•System implementation
•Futureimprovements
•Conclusion
BlueEyesHumanOperatorMonitoringSystem CSU - Data Analysis
• Custom data analyzers - decision tree based classifiers
Example: Operator gesture recognition(simplified)
Vertical Acceleration variance value high
Horizontal Acceleration variance high
Horizontal Acceleration variance high
Y N
Y N YN
No answer “NO”“YES”
•DAU
•CSU
•Motivation
•What isBlue Eyes ?
•Physiological foundations
•Systemdesign
•System implementation
•Futureimprovements
•Conclusion
BlueEyesHumanOperatorMonitoringSystem CSU - Data Analysis
• Custom data analyzers - decision tree based classifiers
• Automatic decision tree coefficients generation - C4.5 based algorithm
Data samplesAssignedcategory
“Yes”
No answer
“Yes”
“No”
No answer
... ... Decision tree coefficients
•DAU
•CSU
•Motivation
•What isBlue Eyes ?
•Physiological foundations
•Systemdesign
•System implementation
•Futureimprovements
•Conclusion
BlueEyesHumanOperatorMonitoringSystem Data security
• Only registered mobile devicescan connect to the system
• Bluetooth connection authentication• Bluetooth connection encryption• Access rights restrictions• Personal and physiological data
encryption
•DAU
•CSU
•Motivation
•What isBlue Eyes ?
•Physiological foundations
•Systemdesign
•System implementation
•Futureimprovements
•Conclusion
BlueEyesHumanOperatorMonitoringSystem Future improvements
• DAU– small CMOS camera to monitor the
operator’s point of gaze– single PCB (SMD technlogy)– low voltage ICs - LiIO batteries power
• CSU– data mining algorithms – advanced database encryption using e.g.
AES algorithm
•Motivation
•What isBlue Eyes ?
•Physiological foundations
•Systemdesign
•System implementation
•Futureimprovements
•Conclusion
BlueEyesHumanOperatorMonitoringSystem Conclusion
• In the near future ,ordinary household devices- such as television , refrigerators ,ovens may be able to do their jobs when we look at them and speak to them.
• Future applications of blue eye technology is limitless
•Motivation
•What isBlue Eyes ?
•Physiological foundations
•Systemdesign
•System implementation
•Futureimprovements
•Conclusion
BlueEyesHumanOperatorMonitoringSystem Thank you for your attention
Any Queries….???
Blue EyesTechnology
•Motivation
•What isBlue Eyes ?
•Physiological foundations
•Systemdesign
•System implementation
•Futureimprovements
•Conclusion