On Human Information Processing in Information Retrieval Alexandre Pereda-Baños (Eurecat), Ioannis Arapakis (Yahoo Labs), Miguel Barreda-Ángeles (Eurecat)

Human Information Processing (HIP)

§  We are not consciously aware of the mental processes determining our behaviour

§  Such unconscious influences reach from basic or low-level mental processes to high-level psychological processes like motivations, preferences, or complex behaviours

Human Information Processing (HIP)

§  The search for information is often led by a human brain §  HIP is the field of study of experimental psychology and cognitive

neuroscience

Psychological Variables

§  The most interesting psychological variables and processes for the study of IR are those related to attentional and emotional phenomena

Selective attention Cognitive

effort / arousal Emotional reactions

Psychophysiological Measures of HIP §  Standardised questionnaires for measuring

perceptual aspects, perceived usability, cognitive working load, or affective

§  Online measures of user behavior and cognitive states that are often unavailable for conscious reports: §  Behavioral §  psychophysiological

Characteristics of Psychological Methods

§  Helpful in unveiling attentional and emotional reactions not consciously available to us

§  Offer high temporal and spatial resolution §  Robust against cognitive biases (e.g., social desirability bias*) §  Always provide “honest” responses §  No direct question to the subject, no direct answer §  The information on the research questions has to be inferred

from the variations on the physiological signals and the way they are related to psychological constructs

* The tendency of survey respondents to answer questions in a manner that will be viewed favorably by others.

Electrodermal Activity (EDA)

§  Changes in conductivity of the skin due to activation of sweat glands by activation of the autonomous nervous system (sympathetic division)

§  Reflects general activation both for attentional and emotional measures (in fact, it is calibrated by having participants perform complex math calculations)

§  It’s the basis of the “truth machine”, though not as effective as fiction has led us to believe…

Electrodermal Activity (EDA)

§  Unconscious Physiological Effects of Search Latency on Users and Their Click Behaviour (SIGIR 2015) •  Although the latency effects did not produce

changes on the self-reported data, their impact on users’ physiological responses is evident

•  Even when short latency increases of under 500ms are not consciously perceived, they have sizeable physiological effects that can contribute to the overall user experience

-0.4-0.20.0 0.2 0.4 0.6 0.81.0 1.2 1.4

1 2 3 4 5 6 7 8 9 10

µS

Time after query onset (in seconds)

0ms500ms750ms1000ms

15.0 15.2 15.4 15.6 15.816.0 16.2 16.4 16.6 16.817.0

0 1 2 3 4 5 6 7 8 9 10 11 12

µS

Time after stimulus onset (in seconds)

Electrodermal Activity (EDA)

§  A large-scale query log analysis ascertained the effect on the clicking behaviour of users and revealed a significant decrease in users’ engagement with the search result page, even at small increases in latency

0

0.05

0.10

0.15

0.20

0 500 750 1000 0

0.5

1.0

1.5

2.0

Fra

ctio

n o

f q

ue

ry p

airs

Clic

k-m

ore

-on

-fa

st/C

lick-

mo

re-o

n-s

low

Latency difference (in milliseconds)

Click-more-on-fastClick-more-on-slowRatio

0

0.05

0.10

0.15

0.20

0 500 750 1000 0

0,5

1.0

1.5

2.0

Fra

ctio

n o

f q

ue

ry p

airs

Clic

k-o

n-f

ast

/Clic

k-o

n-s

low

Latency difference (in milliseconds)

Click-on-fastClick-on-slowRatio

HIP Dynamics §  Human information processing is both serial and parallel §  Cognitive science has provided large amounts of evidence that

conscious information processing is mainly serial §  When processing information in situations that require to shift the

focus of attention between different tasks and/or stimuli, this results in an increase in the effort required to process that information

§  Simon effect

HIP Dynamics (Serial Processing)

HIP Dynamics (Serial Processing) §  Switching tasks §  Try to read the word in odd trials

and name the color on even trials!

Green

Red

Blue

Red

Green

Yellow

HIP Dynamics (Parallel Processing) §  Simon effect: Hit the left key if there is an A on screen and the

right if there is a B

HIP Dynamics (Parallel Processing) §  The effect is still there with crossed hands!

Multimodal Behaviour Analysis §  Behaviour measurements in ecological conditions §  Behaviour understanding through cameras and microphones §  Aggregating various online measures gives an accurate picture

of the user’s experience §  Robust real-time behavior analyses, information that can then be

used for the purpose of research on human behavior and user experience

§  The opportunity is ripe to move beyond experimental laboratory settings, in which response has been traditionally measures, into real large-scale controlled studies

Conclusions §  The use of neuro-physiological methods in IR research is

essential in order to obtain a complete picture of the mental processes underlying user search behaviour

§  The collaboration between psychological and IR research can go far beyond the application of sophisticated measuring methodologies

§  Introduce actual knowledge on the dynamics of human information processing into a real-world testing ground

§  The use of multimodal signals holds the promise of allowing large-scale, controlled studies that will undoubtedly foster the progress of both research fields

Thank you for your attention!

iarapakis

http://www.slideshare.net/iarapakis/sigir15-NeuroIR