towards interaction models derived from eye-tracking data
DESCRIPTION
Presented at Polish IA Summit 2012 in Warsaw on April 19. 2012TRANSCRIPT
Interaction Models Derived From Eye-tracking Data .
Jacek Gwizdka & Michael ColeRutgers University, USA
http://jsg.tel
April 19, 2012
Towards
Eye-tracking?
2
Eye-trackers © Tobii
3
Eye-movement in UX Research
There is a Lot More Eye-tracking Datacan offer UX / HCI / IA
4
Eye-tracking Data
5
State2State1
State3
Patterns
Eye-movement Patterns
New methodology to analyze eye-movement patterns◦Model reading and Measure cognitive effort
◦Correlate with higher-level constructs
user task characteristics, user knowledge, etc.
6
7
Eye-tracking – Fundamentals
8
Reading Model Origins
Based on E-Z Reader model Rayner , Pollatsek, Reichle
◦ Serial reading
◦ Words can be identified in parafovial region
◦ Early lexical access (word familiarity) + Complete lexical processing (word identification)
2o (70px) foveal region parafoveal region
a bit MORE…
9
Two-State Reading Model
◦Filter fixations < 150ms (min time required for lexical processing)
◦Model states characterized by: probability of transitions; number of lexical fixations; duration length of eye-movement trajectory, amount of text covered
ScanRead
1-q
p
1-p
q
a bit MORE…
isolated fixationsfixation
sequences
10
Example Reading Sequence
Fixation sequence: (F F F) F (F F F) F F F F (F F F F F F) FReading model states: R S R S S S S R S
Reading state – R | Scanning state – S
Cognitive Effort Measures of Reading
Reading Speed
Fixation Regression
Perceptual Span
Fixation Duration (“lexical processing excess”)
11
foveal region
a b c d
Perceptual span = Mean(a,b,c,d)
regression
excess
User Study 1: Cognitive Effort and Tasks
12
OBI: advanced obituaryINT: interview preparationCPE: copy editingBIC: background information
N=32
MORE…
Journalists’Information Search
Eye-data and Cognitive Effort Measures
Cognitive effort measuresreading speedmean fixation durationperceptual spantotal fixation regressions
Task complexity by designCopy Editing (CPE) Advance Obituary (OBI)
Search effort task timepages visitedqueries entered
Subjective Task Difficulty
CPE INT BIC OBI
As expected: Copy Editing CPE easiestAdvance Obituary OBI most difficultSig: Kruskal-Wallis χ2 =46.1, p<.0001
13
Eye-data and Task Characteristics
14
Measure Related Task Characteristics
Frequency of reading state transitions
SR bias to readAdvanced obituary and Interview preparation tasks: search for document; task goal not specific
RS bias to scanCopy Editing task: search for segment and task goal specific
ScanRead
1-q
p
1-p
q
MORE…
Copy Editing Interview preparation
User Study 2: Assessing User’s Knowledge
Search in Genomics Domain
15
N=40
MORE…
Rate own domain knowledge
Results: Modeling Domain Knowledge
16
Reading Model features & cognitive effort measures
Eye-tracking Data
Domain knowledge MeSH-based self-ratings
predicted
self-rated
Results: Modeling Domain Knowledge
17
predicted
self-rated
Reading Model features & cognitive effort measures
reading seq length and total durationperceptual spanfixation durationregressions…
Reading Model
Eye-tracking Data Random Forest Model
Domain knowledge MeSH-based self-ratings
m
tk=PDK
m
iii
*5
)*(1
For each user predict
build model
agglomerative hierarchical clustering (Ward’s)
PDK: Participants’ domain knowledge
MORE…
Tobii
Eye-tracking is Coming to Us!
18
Eye-tracker © Tobii | Laptop © Lenovo
From Eye-tracking Data to Interaction Models
Measures derived from eye-movement patterns
Macro use task characteristics, cognitive effort, domain knowledge
Meso reading patterns
Micro eye-gaze positions + timing
19
From Real-time Interactions to Applications
Cognitive Load
Domain Knowledge
Information Relevance
Adapt presentation& content
Enable Interaction(e.g., disabilities)
Task Aspects
Eye-TrackingData
Standard input devices (mouse, keyboard)
other psycho-physiological devices (EEG, SCR, HRV)
Better understand interaction
Micro-level
Macro-level
Applications
Cognitive Load Model
ReadingModel
Task Model
Models
20
…
…
…
21
Thank You! Dziekuje!
Funding: Google, HP, IMLS (now funded by IMLS CAREER)Collaborators: Drs. Nicholas Belkin, Art Chaovalitwongse (U Wash), Xiangmin Zhang,
Ralf Bierig (Post Doc); PhD students: Michael Cole (co-author), Chang Liu, Jingjing Liu, Irene Lopatovska
+ many Master and undergraduate students …
Acknowledgements:
22
Pytania?
More info & contact http://jsg.tel