Integrating Eye Tracking Data with Physiological Measurements
Dr. Meike MischoSensoMotoric Instruments
Frazer FindlayBIOPAC Systems, Inc.
Dr. Arnd RoseSensoMotoric Instruments
Sponsored by:
InsideScientific is an online educational environment designed for life science researchers. Our goal is to aid in
the sharing and distribution of scientific information regarding innovative technologies, protocols, research
tools and laboratory services.
Integrating Eye Tracking Data with Physiological Measurements
Dr. Meike MischoProduct Manager, SensoMotoric Instruments (SMI)
© SensoMotoric Instruments GmbH and BIOPAC Systems, Inc. 2016 www.smivision.com www.biopac.com
Integrating
Eye Tracking Data
with
Physiological
Measurements
4
© SensoMotoric Instruments GmbH and BIOPAC Systems, Inc. 2016 www.smivision.com www.biopac.com
Eye Positions and Eye Movements
Cognitive processing
Visual Processing
Wayfinding
Social dynamics
…
Eye Tracking: What are we looking at?
5
Screen-based Mobile Virtual Reality
© SensoMotoric Instruments GmbH and BIOPAC Systems, Inc. 2016 www.smivision.com www.biopac.com
Physiological Measurements: Measure Body Functions
6
EMG
Muscle Activity
e.g. Muscle Conditioning
ECG
Heart Activity
e.g. AnxietyEDA
Skin Conductance
e.g. Alertness
EEG
Brain Activity
e.g. Cognitive
Processes
and many more
EGG, Pulse, Respiration,
Temperature, …
© SensoMotoric Instruments GmbH and BIOPAC Systems, Inc. 2016 www.smivision.com www.biopac.com
Map cause-effect relationships with integrated measurements
7
Which parts in a task
cause cognitive load?
EEG + Eye Tracking
What situations cause
anxiety?
EDA + Eye Tracking
What are strategies to deal
with complex situations?
ECG + Eye Tracking
How do we plan and
execute motor tasks?
EMG + Eye Tracking
Eye Tracking - Cause
Physiology - Effect
© SensoMotoric Instruments GmbH and BIOPAC Systems, Inc. 2016 www.smivision.com www.biopac.com
How to synchronize data streams
8
🕓 12:01:09🕓 11:59:42
Eye Tracking Recording
Physiology Recording
Time of recorder12:00:00 13:00:00
© SensoMotoric Instruments GmbH and BIOPAC Systems, Inc. 2016 www.smivision.com www.biopac.com 9
🕓 12:01:09🕓 11:59:42Issues:
1.
2.
Eye Tracking Recording
Physiology Recording
Recording time00:00:00 01:00:00
How to synchronize data streams
© SensoMotoric Instruments GmbH and BIOPAC Systems, Inc. 2016 www.smivision.com www.biopac.com 10
🕓 12:01:09🕓 11:59:42Issues:
1.
2.
Eye Tracking Recording
Physiology Recording
Recording time00:00:00
01:06:15:44501:06:15:440
01:00:00
How to synchronize data streams
© SensoMotoric Instruments GmbH and BIOPAC Systems, Inc. 2016 www.smivision.com www.biopac.com 11
🕓 12:01:09🕓 11:59:42 Issues:
1.
2.Usual workflow:
1.
2.
3.
Eye Tracking Recording
Physiology Recording
Recording time00:00:00
01:06:15:44501:06:15:440
01:00:00
How to synchronize data streams
© SensoMotoric Instruments GmbH and BIOPAC Systems, Inc. 2016 www.smivision.com www.biopac.com 12
🕓 12:01:09🕓 11:59:42
How to synchronize data streams
Eye Tracking Recording
Physiology Recording
Recording time00:00:00 01:00:00
© SensoMotoric Instruments GmbH and BIOPAC Systems, Inc. 2016 www.smivision.com www.biopac.com
🕓 12:01:09🕓 11:59:42
13
Usual workflow:
1.
2.
3.
1. Send Markers• TTL (Parallel Port)
• Network (TCP/IP, UDP)
2. Reset Clocks
3. Use Physiological Events• Blinks in EEG/Eye Tracking
How to synchronize data streams
Eye Tracking Recording
Physiology Recording
Recording time00:00:00 01:00:00
© SensoMotoric Instruments GmbH and BIOPAC Systems, Inc. 2016 www.smivision.com www.biopac.com
Joint recordings with SMI/BIOPAC
14
Presentation by
Design & Record Analyze
Proven solutions for reliable
synchronization tailored to use cases
Click on the
various product
images to learn
more
Integrating Eye Tracking Data with Physiological Measurements
Frazer Findlay CEO, BIOPAC Systems, Inc.
BIOPAC Systems, Inc.
We provide scientific tools for measuring physiology any time, any where from any subject to academic
researchers and industry scientists who want to make great discoveries.
Where we came from…
Research was typically restricted to
the lab with little opportunity for
experimental complexity because the
tools just weren’t sophisticated
enough – the technology wasn’t
available for the things that we are
discussing today…
Image courtesy of www.binarysleep.com
Physiology Anywhere and Everywhere
Exercise Play
WorkLab Sleep
© SensoMotoric Instruments GmbH and BIOPAC Systems, Inc. 2016 www.smivision.com www.biopac.com18
Key Concepts
Aligning physiological data with eye tracking data
Aligning all data with the stimulus events
Blood Pressure
Electrodermal Activity
Left Pupil Diameter Y
Left Pupil Diameter X
Left Pupil Width
Physiological Data with Eye Tracking Data
www.smivision.com www.biopac.com20
© SensoMotoric Instruments GmbH and BIOPAC Systems, Inc. 2016
3.311 seconds offset from the event mark
Poorly Aligned Data
www.smivision.com www.biopac.com 21© SensoMotoric Instruments GmbH and BIOPAC Systems, Inc. 2016
1.279 second offset from the event mark
Correctly Aligned Data
www.smivision.com www.biopac.com22
© SensoMotoric Instruments GmbH and BIOPAC Systems, Inc. 2016
Stimulus Event Marks
www.smivision.com www.biopac.com 23© SensoMotoric Instruments GmbH and BIOPAC Systems, Inc. 2016
Summary
Recording great data
Recording the signals that are important to the study
Recording data where you want
Aligning physiological data with eye tracking data
Aligning all data with the stimulus events
Laboratory Solution
MP150 System
BioNomadix Wireless Transmitter
RED250Mobile Eye Tracker
Isolated Stimulus Presentation Interface Module
BioNomadix Receiver
Laboratory Solution
www.smivision.com www.biopac.com 25© SensoMotoric Instruments GmbH and BIOPAC Systems, Inc. 2016
Demonstration of AcqKnowledge Software
View Recording
Eye Tracking with Mobile Subjects
www.smivision.com www.biopac.com 27© SensoMotoric Instruments GmbH and BIOPAC Systems, Inc. 2016
Today...wireless, wearable physiology – anywhere!
Combine SMI Glasses with BIOPAC BioNomadix
Integrating Eye Tracking Data with Physiological Measurements
Dr. Arnd Rose Product Manager, SensoMotoric Instruments (SMI)
30
© SensoMotoric Instruments GmbH and BIOPAC Systems, Inc. 2016 www.smivision.com www.biopac.com
SMI ETG Integration with BIOPAC BioNomadix Logger
31
SMI Eye Tracking Glasses
(ETG)
BIOPAC BioNomadix Logger
© SensoMotoric Instruments GmbH and BIOPAC Systems, Inc. 2016 www.smivision.com www.biopac.com
SMI ETG + BioNomadix Logger augments diverse use cases
Human Factors / Ergonomics: Workplace optimization
o Combining information about physiological state, perception and
performance to understand and improve human interfaces.
Psychology: Human to human interaction
o Add arousal information to perception.
o Add information of visual perception to interpretation of physiological data.
Market Research: Decision making in public places
o Add arousal information to improve understanding of decision making.
32
© SensoMotoric Instruments GmbH and BIOPAC Systems, Inc. 2016 www.smivision.com www.biopac.com
Use Case: HMI research for driver assistance system
Goal:
o Evaluate stress level of drivers while operating
different assistance systems during driving.
Test design:
o A-B Testing: Drivers drive a given route in real
traffic while processing different tasks in the
test car and a control vehicle.
33
© SensoMotoric Instruments GmbH and BIOPAC Systems, Inc. 2016 www.smivision.com www.biopac.com
Equipment setup
BIOPAC
Logger
SMI
Smart
Recorder
Initial clock synchronization
BIOPAC Sensors
-EDA
-PPG
-ECG
-Respiration
BIOPAC BioNomadix
Transmitter
1. PPG & EDA
2. RSP & ECG
BIOPAC BioNomadix
Logger
SMI Eye Tracking
Glasses and Smart
Recorder
34
© SensoMotoric Instruments GmbH and BIOPAC Systems, Inc. 2016 www.smivision.com www.biopac.com
Time of Day Synchronization
BIOPAC
clock
SMI
clock
Synchronization
of internal clocks
35
Both systems will start recording
independent of each other, but their
“time of day” clocks are in sync…
© SensoMotoric Instruments GmbH and BIOPAC Systems, Inc. 2016 www.smivision.com www.biopac.com
Data are aligned in AcqKnowledge Software
BIOPAC
clock
SMI
clock
36
Synchronization
of internal clocks
© SensoMotoric Instruments GmbH and BIOPAC Systems, Inc. 2016 www.smivision.com www.biopac.com
Hands-on process
Preparation Setup Recording Analysis
37
© SensoMotoric Instruments GmbH and BIOPAC Systems, Inc. 2016 www.smivision.com www.biopac.com
ETG & Smart Recorder: System Setup
Connect ETG to Smart
Recorder
Turn on the Smart
Recorder
Preparation Setup Recording Analysis
38
© SensoMotoric Instruments GmbH and BIOPAC Systems, Inc. 2016 www.smivision.com www.biopac.com
BioNomadix: System Setup
Turn on BioNomadix
logger
Connect transmitters
39
Preparation Setup Recording Analysis
© SensoMotoric Instruments GmbH and BIOPAC Systems, Inc. 2016 www.smivision.com www.biopac.com
Synchronize internal clocks of devices
Synchronization tool
sets both internal
clocks to the same
time
40
Preparation Setup Recording Analysis
BIOPAC
Logger
SMI
Smart
Recorder
© SensoMotoric Instruments GmbH and BIOPAC Systems, Inc. 2016 www.smivision.com www.biopac.com 41
Setup Preparation Recording Analysis
© SensoMotoric Instruments GmbH and BIOPAC Systems, Inc. 2016 www.smivision.com www.biopac.com 42
View Recording
© SensoMotoric Instruments GmbH and BIOPAC Systems, Inc. 2016 www.smivision.com www.biopac.com
Data Export from SMI ETG
Direct Gaze Data is
saved to the Smart
Recorder’s Micro SD
Card
Export Data to a PC by
Removing SD Card:
Data Storage Location
and File Structure
43
Setup Preparation Recording Analysis
© SensoMotoric Instruments GmbH and BIOPAC Systems, Inc. 2016 www.smivision.com www.biopac.com
Data Import from BioNomadix logger
Connect logger by USB
to analysis PC
Start AcqKnowledge SW
Import Data to
AcqKnowledge
Setup Preparation Recording Analysis44
© SensoMotoric Instruments GmbH and BIOPAC Systems, Inc. 2016 www.smivision.com www.biopac.com
Data Integration and Import
Enable Media Playback
Open the recorded ETG
video
Video plays in parallel to
data view
Setup Preparation Recording Analysis45
© SensoMotoric Instruments GmbH and BIOPAC Systems, Inc. 2016 www.smivision.com www.biopac.com
46
View Recording
© SensoMotoric Instruments GmbH and BIOPAC Systems, Inc. 2016 www.smivision.com www.biopac.com
Why and how to perform co-registration?
Eye tracking and physiological data measurements
reveal cause and effect relationships, e.g.:
Which situations cause anxiety?
What increases cognitive load?
How to synchronize data in diverse settings:
Methods for synchronization
Laboratory setup with TTL trigger synchronization
Real life environment with clock synchronization
Eye Tracking - Cause
Physiology - Effect
🕓 12:01:09🕓 11:59:42
© SensoMotoric Instruments GmbH and BIOPAC Systems, Inc. 2016 www.smivision.com www.biopac.com
Efficient workflow for co-registration
SMI/BIOPAC integration provides an efficient co-
registration study workflow:
Easy to use hardware and software for both mobile
and lab environments
Efficient co-recording of eye tracking, physiological
data and stimulus events
Automated and precise synchronization via plug-
and-play integration of analysis software
Please contact us to learn more!
Thank You!
Dr. Meike Mischo Frazer Findlay Dr. Arnd Rose
If you have questions for the presenters please contact them by email.
For additional information on the solutions presented in this webinar please visit:
http://www.biopac.com http://www.smivision.com
BIOPAC Systems: [email protected]
SensoMotoric Instruments: [email protected]
Access the full Recording here