wearable technology: the wave of the futurefiles.meetup.com/1698110/dehzangi - presentation.pdf ·...
TRANSCRIPT
Omid Dehzangi
Computer and Information Science
University of Michigan - Dearborn
Wearable Sensing and Signal Processing Lab
Wearable Technology: the wave of the future
Outline
Introduction to wearable technology
Vision and mission
Application and high level model design
Wearable platform design and development
My research contributions
Brain-computer interface
Activity of daily living (ADL) monitoring
My current research plans
2
Technology Trends
Transistors
Digital Processing
Brought to homes
3
Analog computer Personal Computers Wearable Computers Today’s Computers
Smaller
Slimmer
Faster
Hand held
Smarter
Hands free
Natural Interface
Wearable Technology
ABI Research has projected that by 2016, wearable wireless device sales will reach more than 100 million devices annually.
The market for wearable sports, fitness, and healthcare monitoring devices cover 80% of it.
The market for wearable technologies in healthcare "is projected to exceed $2.9 billion in 2016 (at least half of all wearable technology)
4
Photo courtesy of http://www.phonearena.com/
Vibrotactile Modules
Wearable Computers
5
Phantom Photo Courtesy of
SenseGraphics
Haptics Deep Brain Stimulation Photo Courtesy of
mindmodulation.com
GUI-based feedback
Feedback
Sensors Dry-contact
EEG
Inertial
Sensors
Galvanic Skin
Response
Flex, Pressure and Piezo-electric
Sensors
Wearable Computers
6
Sensors
Processing Unit
Communication
Information Fusion
Prediction/Detection
Data Analytics
Big data analysisData miningMachine learningPredictive modelingStatistical analysis
Signal Processing
Feedback
Outline
Introduction to wearable technology
Vision and mission Application and high level model design
Wearable platform design and development
7
Research Vision
Applications
Model design
Algorithms
and analytics
System
integration
Technologies
Demonstrate the linkage between discovery
and societal benefit
Validate real pains and necessities and identify
effective high level solutions
Design and develop in multiple technical levels
Resolve upcoming challenges in practice
Generate transitioning technologies
Wearable platform
9
Wireless Health Ubiquitous monitoring and intervention for the applications of health-care and wellness
9
Courtesy of Misha Pavel, Program Director, National Science Foundation
Outline
Introduction to wearable technology
Vision and mission
Application and high level model design
Wearable platform design and development
My current research contributions
Brain-computer interface
Activity monitoring and motion detection
10
WEARABLE BRAIN COMPUTER INTERFACE
Application Case Study
11
Applications
Brain Computer Interface
• Brain Computer Interface
– Provide a non-muscular avenue for the user to communicate with others and to control external devices
– Infer user’s intentions using brain activities
• Applications
– Assist locked-in individuals to interact with cyber and physical system
– Gaming
– Diagnosis and treatment for neurological disorder
12
Wearable EEG Systems • Smaller form factor (size
of a credit card vs. bulky amps)
• Quicker setup time (seconds vs. 30 mins)
• Faster software training (5 mins vs. 30 mins)
• Quicker EEG signal detection (seconds vs. minutes)
• No need for EEG tech
13
Custom-designed mobile EEG-based BCI Dry-contact electrodes
Low-noise front-end (ADS1299)
Low power processing (MSP430)
Low component count
Bluetooth low energy (TI BLE)
communication module
Wearable EEG-Based BCI
14
Wearable BCI Units
15
Canonical Correlation Analysis(CCA)
16
Picture taken from ref.1
Video
17
USING GAIT AND SWAY BIOFEEDBACK TO REDUCE FALLS IN THE ELDERLY
17
Applications
ACTIVITY OF DAILY LIVING MONITORING
• Dehzangi, Omid, Biggan, John, Birjandtalab Golkhatmi, Javad, Ray, Christopher, Jafari,
Roozbeh, “An Inertial Sensor-Based Method for Early Detection and Prevention of
Excessive Sway in Older Adults via Gait Analysis and Vibrotactile Biofeedback”, Gait &
Posture journal.
• Dehzangi, Omid, Zhao, Zheng, Biggan, John, Ray, Christofer, Jafari, Roozbeh, “The
Impact of Vibrotactile Biofeedback on the Excessive Walking Sway and the Postural
Control in Elderly”, Wireless Health 2013, November 1-3, Baltimore, Maryland, 2013.
Application Case Study
Postural control and gait analysis
18
Sway Biofeedback for Fall Prevention
Fall is a considerable health concern in the elderly
Wearable kinematic biofeedback system to detect pre-cursors of falls based on the sway of the upper body and other gait parameters, and activate biofeedback
19
20
Hardware Architecture
Laptop UART
BLE transceiver
Microprocessor
Motion Sensor:
Gyro &
Accelerometer BLE
I2C
21
Software Architecture
Accelero
meter
X,Y
,Z
Sensor
CalibrationDrift
Detection
PI controller
DCM
Self-ad
aptiv
e
Angle T
hresh
old
Settin
g
Calibrated
Accelerometer
X,Y,Z
Calibrated
Gyroscope
X,Y,Z
Feedback
loop
Euler Angles
(roll, pitch)
X,Y
,Z
X,Y
,ZA
ccelerom
eter
X,Y
,Z
Gyro
scope
The developed system
22
(a) (b)
(a) Our designed wearable low-power motion sensor board,
(b) Our biofeedback system, consisting of two motion sensor boards for the chest and the ankle along with the vibratory feedback modules.
Subjects:
24 older adults (age: M = 75.5, SD = 4.32 years; 10 females)
Procedure:
23
Experiments
24
Mean difference in the sway range.
The test Control Experimental P-value
Difference in the sway range
0.59±1.77 -0.60±0.63 0.04
The results of the statistical test on the sway range
Results and Analysis
25
Identification of the gait phases on the ACC X readings
Gait phase analysis
Initial sway Mid sway Terminal sway Mid stance
The selected phases of a gait cycle.
26
DTW extracted strides based on ACC X readings (Experimental)
Gait phase analysis
0 5 10 15 20 25-2
-1.5
-1
-0.5
0
0.5
1
1.5x 10
4
Sample
AC
C X
re
ad
ing
0 5 10 15 20 25 30-2
-1.5
-1
-0.5
0
0.5
1
1.5
2x 10
4
AC
C X
re
ad
ing
Sample
DTW extracted strides based on ACC X readings (Control)
27
Mean difference in the variance of the gait phases between pre- and post- training.
Gait phase analysis
The gait phases Control Experimental P-value
Initial sway 0.17±0.62 0.40±0.15 0.09
Mid sway -1.54±1.72 -0.03±0.48 0.08
Terminal sway -1.29±1.04 0.038±1.01 0.05
Mid stance -0.18±0.89 -0.07±1.14 0.27
The results of the Chi-square test on the gait phases
Initial sway Mid sway Terminal sway Mid stance
Outline
Introduction to wearable technology
Vision and mission
Application and high level model design
Wearable platform design and development
My current research contributions
Brain-computer interface
Activity of daily living (ADL) monitoring
My Current Research Plans
28
WEARABLE DRIVER MONITORING
Application Case Study
29
Applications
Goal
• To form relationships between biological state of the driver with his/her driving behavior
37
31
Motivation:
Multi-Modal Driver Monitoring and Modeling via Heterogeneous Wearable Body Sensor Network
System
photograph:
a) b) c)
Integration of heterogeneous wearable monitoring technology, on-board sensing units, and wireless
networking capabilities : a) The full body sensor network, b) the portable EEG system, c) the OBD-II device
Body sensor networks are capable of generating a reliable human state model
Hypotheses:
1. Minimally intrusive: Driver behavior is not affected by
the devices that are used to acquire the necessary
biomedical markers
2. Comprehensive: the system will extract the data
collected from a large number of heterogeneous
sensors and correlate the various readings for earlier
detection
3. Ubiquitous and remotely available: The collected
measurements will be transmitted to a remote location
for longitudinal analysis and discover association in a
long term
4. Real-time responsive: The information will be
accessible in an online fashion to enable real-time
processing and decision making
5. User friendly: Suitable user interface and
visualization tools will be in place for a human user to
be able to interpret the acquired information
Hypothesis 1: Specific driver mental and
physical states can generate abnormal driving
behaviors and a high level of driving impairment.
Hypothesis 2: Driver biological states will have
an impact on his/her biometric measures while
driving. Biometric markers that correspond to
changes in performance of the impaired driver
subjects will aid in explaining the underlying
impact on driving outcome.
Hypothesis 3: There are signature patterns in
the biometric readings from the normal behavior
of the driver that can be non-invasively extracted
and employed for control, identification &
authentication, and interaction with other smart
infrastructures.
The Proposed Platform:
Multi-Modal Driver Monitoring and Modeling via Heterogeneous Wearable Body Sensor Network
39
Multi-Modal Driver Monitoring and Modeling via Heterogeneous Wearable Body Sensor Network
Real-time Processing
EEG
ECG
OBD-II
DA
Q-F
rontE
nd
Data
Min
ing
Sensors In-Vehicle Mobile Device
Long-term analysis
Data
Base
Big
Da
ta
An
aly
sis
Backend Processing
Bio-feedback
Data
Vis
ua
liza
tion
GSR Data
Ma
trix
User Interface
Driver
DA
Q-B
ack
En
d
GPS
TrafficB
lue
To
oth
Mo
bile
Ne
two
rk
40
Platform User Interface
34
Characterizing Driver Distraction
30
Characterizing Driver Distraction:
Two rounds of driving:
1. Non-peak traffic period,
2. Peak traffic period
Objective:
Investigate the effect of the road condition
on the driver distraction
Hypothesis:
Theta and beta power increase in the EEG
spectrum is related to distraction effects
45
Characterizing Driver Distraction
30
Comparison of total theta and beta power (dB×103) in
the subjects averaged EEG in frontal component
between round 1 and round 2
Subjects Round 1 Round 2
theta beta theta beta
Sbj 1 1.8 1.2 1.5 1.2
Sbj 2 2.8 1.6 1.7 1.3
Sbj 3 2.5 1.5 1.3 1.1
Avg 2.33 1.5 1.48 1.27
47
Characterizing Driver Distraction
48
-0.1 -0.05 0 0.05 0.1 0.15-0.1
-0.05
0
0.05
0.1
0.15
x
y
pdf(obj,[x,y])
-0.1 -0.05 0 0.05 0.1 0.15-0.1
-0.05
0
0.05
0.1
0.15
x
y
pdf(obj,[x,y])
the effect of the driving condition on the driver distraction
-0.1 -0.05 0 0.05 0.1 0.15-0.1
-0.05
0
0.05
0.1
0.15
x
y
pdf(obj,[x,y])
-0.1 -0.05 0 0.05 0.1 0.15-0.1
-0.05
0
0.05
0.1
0.15
x
y
pdf(obj,[x,y])
38
Ideas to Pursue
Towards proactive driver monitoring and safety platform as
advancement in automated passenger vehicle infrastructure.
Connect the vehicle occupants to the loop via development of D2V & D2I
in automated vehicles to improve occupant safety, performance, health &
wellness.
The connected vehicle infrastructure will associate the driver with the
smart city and/or smart home infrastructure to optimize his/her daily
operations.
Driver identification and authentication is an important outcome which
will be performed non-invasively via extracting the signature biometrics
from the normal behavior of the driver.
51
Thanks for your attention
39