integrating eye tracking data with physiological measurements

Integrating Eye Tracking Data with Physiological Measurements

Dr. Meike MischoSensoMotoric Instruments

Frazer FindlayBIOPAC Systems, Inc.

Dr. Arnd RoseSensoMotoric Instruments

Sponsored by:

InsideScientific is an online educational environment designed for life science researchers. Our goal is to aid in

the sharing and distribution of scientific information regarding innovative technologies, protocols, research

tools and laboratory services.

http://insidescientific.com/



Dr. Meike MischoProduct Manager, SensoMotoric Instruments (SMI)

http://www.smivision.com/en/gaze-and-eye-tracking-systems/home.html


© SensoMotoric Instruments GmbH and BIOPAC Systems, Inc. 2016 www.smivision.com www.biopac.com

Integrating

Eye Tracking Data

with

Physiological

Measurements

4



http://www.biopac.com/



Eye Positions and Eye Movements

Cognitive processing

Visual Processing

Wayfinding

Social dynamics

…

Eye Tracking: What are we looking at?

5

Screen-based Mobile Virtual Reality


Physiological Measurements: Measure Body Functions

6

EMG

Muscle Activity

e.g. Muscle Conditioning

ECG

Heart Activity

e.g. AnxietyEDA

Skin Conductance

e.g. Alertness

EEG

Brain Activity

e.g. Cognitive

Processes

and many more

EGG, Pulse, Respiration,

Temperature, …


Map cause-effect relationships with integrated measurements

7

Which parts in a task

cause cognitive load?

EEG + Eye Tracking

What situations cause

anxiety?

EDA + Eye Tracking

What are strategies to deal

with complex situations?

ECG + Eye Tracking

How do we plan and

execute motor tasks?

EMG + Eye Tracking

Eye Tracking - Cause

Physiology - Effect


How to synchronize data streams

8

🕓 12:01:09🕓 11:59:42

Eye Tracking Recording

Physiology Recording

Time of recorder12:00:00 13:00:00





© SensoMotoric Instruments GmbH and BIOPAC Systems, Inc. 2016 www.smivision.com www.biopac.com 9

🕓 12:01:09🕓 11:59:42Issues:

1.

2.



Recording time00:00:00 01:00:00







🕓 12:01:09🕓 11:59:42Issues:

1.

2.



Recording time00:00:00

01:06:15:44501:06:15:440

01:00:00







🕓 12:01:09🕓 11:59:42 Issues:

1.

2.Usual workflow:

1.

2.

3.



Recording time00:00:00

01:06:15:44501:06:15:440

01:00:00







🕓 12:01:09🕓 11:59:42










🕓 12:01:09🕓 11:59:42

13

Usual workflow:

1.

2.

3.

1. Send Markers• TTL (Parallel Port)

• Network (TCP/IP, UDP)

2. Reset Clocks

3. Use Physiological Events• Blinks in EEG/Eye Tracking






Joint recordings with SMI/BIOPAC

14

Presentation by

Design & Record Analyze

Proven solutions for reliable

synchronization tailored to use cases

Click on the

various product

images to learn

more



http://www.smivision.com/en/gaze-and-eye-tracking-systems/products/begaze-analysis-software.html


http://www.smivision.com/en/gaze-and-eye-tracking-systems/products/experiment-center-software.html


http://www.eyetracking-glasses.com/











Frazer Findlay CEO, BIOPAC Systems, Inc.



BIOPAC Systems, Inc.

We provide scientific tools for measuring physiology any time, any where from any subject to academic

researchers and industry scientists who want to make great discoveries.



Where we came from…

Research was typically restricted to

the lab with little opportunity for

experimental complexity because the

tools just weren’t sophisticated

enough – the technology wasn’t

available for the things that we are

discussing today…

Image courtesy of www.binarysleep.com

Physiology Anywhere and Everywhere

Exercise Play

WorkLab Sleep

© SensoMotoric Instruments GmbH and BIOPAC Systems, Inc. 2016 www.smivision.com www.biopac.com18

Key Concepts

Aligning physiological data with eye tracking data

Aligning all data with the stimulus events

Blood Pressure

Electrodermal Activity

Left Pupil Diameter Y

Left Pupil Diameter X

Left Pupil Width

Physiological Data with Eye Tracking Data

www.smivision.com www.biopac.com20

© SensoMotoric Instruments GmbH and BIOPAC Systems, Inc. 2016





3.311 seconds offset from the event mark

Poorly Aligned Data

www.smivision.com www.biopac.com 21© SensoMotoric Instruments GmbH and BIOPAC Systems, Inc. 2016





1.279 second offset from the event mark

Correctly Aligned Data

www.smivision.com www.biopac.com22

© SensoMotoric Instruments GmbH and BIOPAC Systems, Inc. 2016





Stimulus Event Marks






Summary

Recording great data

Recording the signals that are important to the study

Recording data where you want

Aligning physiological data with eye tracking data

Aligning all data with the stimulus events

Laboratory Solution

MP150 System

BioNomadix Wireless Transmitter

RED250Mobile Eye Tracker

Isolated Stimulus Presentation Interface Module

BioNomadix Receiver

Laboratory Solution


Demonstration of AcqKnowledge Software

View Recording

https://attendee.gotowebinar.com/register/5859565018985420546

Eye Tracking with Mobile Subjects


Today...wireless, wearable physiology – anywhere!

Combine SMI Glasses with BIOPAC BioNomadix


Dr. Arnd Rose Product Manager, SensoMotoric Instruments (SMI)

30




SMI ETG Integration with BIOPAC BioNomadix Logger

31

SMI Eye Tracking Glasses

(ETG)

BIOPAC BioNomadix Logger


SMI ETG + BioNomadix Logger augments diverse use cases

Human Factors / Ergonomics: Workplace optimization

o Combining information about physiological state, perception and

performance to understand and improve human interfaces.

Psychology: Human to human interaction

o Add arousal information to perception.

o Add information of visual perception to interpretation of physiological data.

Market Research: Decision making in public places

o Add arousal information to improve understanding of decision making.

32


Use Case: HMI research for driver assistance system

Goal:

o Evaluate stress level of drivers while operating

different assistance systems during driving.

Test design:

o A-B Testing: Drivers drive a given route in real

traffic while processing different tasks in the

test car and a control vehicle.

33


Equipment setup

BIOPAC

Logger

SMI

Smart

Recorder

Initial clock synchronization

BIOPAC Sensors

-EDA

-PPG

-ECG

-Respiration

BIOPAC BioNomadix

Transmitter

1. PPG & EDA

2. RSP & ECG

BIOPAC BioNomadix

Logger

SMI Eye Tracking

Glasses and Smart

Recorder

34


Time of Day Synchronization

BIOPAC

clock

SMI

clock

Synchronization

of internal clocks

35

Both systems will start recording

independent of each other, but their

“time of day” clocks are in sync…






Data are aligned in AcqKnowledge Software

BIOPAC

clock

SMI

clock

36

Synchronization

of internal clocks






Hands-on process

Preparation Setup Recording Analysis

37






ETG & Smart Recorder: System Setup

Connect ETG to Smart

Recorder

Turn on the Smart

Recorder


38






BioNomadix: System Setup

Turn on BioNomadix

logger

Connect transmitters

39







Synchronize internal clocks of devices

Synchronization tool

sets both internal

clocks to the same

time

40


BIOPAC

Logger

SMI

Smart

Recorder


Setup Preparation Recording Analysis


View Recording



Data Export from SMI ETG

Direct Gaze Data is

saved to the Smart

Recorder’s Micro SD

Card

Export Data to a PC by

Removing SD Card:

Data Storage Location

and File Structure

43

Setup Preparation Recording Analysis






Data Import from BioNomadix logger

Connect logger by USB

to analysis PC

Start AcqKnowledge SW

Import Data to

AcqKnowledge

Setup Preparation Recording Analysis44






Data Integration and Import

Enable Media Playback

Open the recorded ETG

video

Video plays in parallel to

data view

Setup Preparation Recording Analysis45






46

View Recording



Why and how to perform co-registration?

Eye tracking and physiological data measurements

reveal cause and effect relationships, e.g.:

Which situations cause anxiety?

What increases cognitive load?

How to synchronize data in diverse settings:

Methods for synchronization

Laboratory setup with TTL trigger synchronization

Real life environment with clock synchronization

Eye Tracking - Cause

Physiology - Effect

🕓 12:01:09🕓 11:59:42


Efficient workflow for co-registration

SMI/BIOPAC integration provides an efficient co-

registration study workflow:

Easy to use hardware and software for both mobile

and lab environments

Efficient co-recording of eye tracking, physiological

data and stimulus events

Automated and precise synchronization via plug-

and-play integration of analysis software

Please contact us to learn more!

Thank You!

Dr. Meike Mischo Frazer Findlay Dr. Arnd Rose

If you have questions for the presenters please contact them by email.

For additional information on the solutions presented in this webinar please visit:

http://www.biopac.com http://www.smivision.com

BIOPAC Systems: [email protected]

SensoMotoric Instruments: [email protected]

Access the full Recording here








integrating eye tracking data with physiological measurements

Science