perception of time -...

Perception of TimePerception of Time

Niels Taatgen & Hedderik van RijnNiels Taatgen & Hedderik van RijnNiels Taatgen & Hedderik van RijnNiels Taatgen & Hedderik van RijnDepartment of Artificial Intelligence/PsychologyDepartment of Artificial Intelligence/PsychologyUniversity of GroningenUniversity of Groningen

artificialintelligence

cognitivemodeling


cognitivemodeling

University of GroningenUniversity of Groningen

Estimating time is a component Estimating time is a component of skilled performanceof skilled performance


cognitivemodeling

OverviewOverview

A model of time estimation within a A model of time estimation within a cognitive architecturecognitive architectureModels of standard estimation Models of standard estimation experiments and bisectionexperiments and bisectionRole of attention: DualRole of attention: Dual--task timing task timing tasktaskLogarithmic or Linear: Adding intervalsLogarithmic or Linear: Adding intervalsg gg gRole of memory: Dikes and RiversRole of memory: Dikes and Rivers


cognitivemodeling

Time estimation: many modelsTime estimation: many models

Attentional gating Attentional gating (Zakay, Block)(Zakay, Block)

Pacemaker Gate

StartSignal

AccumulatorGate

Attention

Memory

( y )( y )Signal

Comparison

Pacemaker Gate Accumulator

Memory

Internal clock (Gibbon, Meck)

StartSignal

Memory

Comparison


cognitivemodeling

Comparison

Embed time estimation in a Embed time estimation in a general cognitive architecturegeneral cognitive architecture

Pacemaker Gate

StartSignal

AccumulatorAccumulator

Memory

ComparisonDeclarative Module

Retrieval BufferRetrieval BufferGoal Buffer

Matching

Selection

ExecutionProd

uctio

nsPr

oduc

tions

Problem BufferProblem BufferPacemaker Gate

S


Visual Module Manual Module

Manual BufferVisual BufferStart

SignalTaatgen, N. A., Rijn, H. v., & Anderson, J. R. (2007). An Integrated Theory of Prospective


cognitivemodeling

External WorldTime Interval Estimation: The Role of Cognition, Attention and Learning. Psychological Review, 114(3), 577-598.

AttentionAttentionTemporal Module

ACTACT--R basically R basically conforms with conforms with

Declarative Module


Accumulator

e po a odu e

central bottleneck central bottleneck theories (e.g., theories (e.g., Pashler seePashler see

Matching

Selection

ExecutionProd

uctio

nsPr

oduc

tions

Problem BufferProblem Buffer

Pashler, see Pashler, see Salvucci & Salvucci & Taatgen 2008)Taatgen 2008)

PP


Manual BufferVisual Buffer

Taatgen, 2008)Taatgen, 2008)External World


cognitivemodeling

LearningLearningTemporal Module

ACTACT--R uses a form of R uses a form of instance learning instance learning (Logan)(Logan)

Declarative Module


Accumulator

e po a odu e

( g )( g )•• Experiences with an Experiences with an

interval are stored in interval are stored in memorymemory

Matching

Selection

ExecutionProd

uctio

nsPr

oduc

tions

Problem BufferProblem Buffer

•• Memory is subject to Memory is subject to decaydecay

ACTACT--R uses rule R uses rule l i l il i l i

PP


Manual BufferVisual Buffer

learning: can explain learning: can explain how judging a particular how judging a particular interval can be interval can be automatedautomated

External World


cognitivemodeling

automatedautomated

Underlying principle of the Underlying principle of the Timing ModuleTiming Module

WeberWeber’’s Laws LawHow to we perceive changes in stimuli?How to we perceive changes in stimuli?How to we perceive changes in stimuli?How to we perceive changes in stimuli?

dp = k dS

The change we perceive is proportional The change we perceive is proportional

pS

to the physical change divided by the to the physical change divided by the magnitude of the stimulus itselfmagnitude of the stimulus itself


cognitivemodeling

WeberWeber’’s law is broadly s law is broadly applicableapplicable

After integration, this becomesAfter integration, this becomes

p k lnS+CItIt’’s applicable tos applicable to

p = k lnS+CItIt s applicable tos applicable to•• Changes in weightChanges in weight•• Differences in light intensity and volumeDifferences in light intensity and volumeDifferences in light intensity and volumeDifferences in light intensity and volume•• How people judge numbersHow people judge numbers•• And also how we perceive timeAnd also how we perceive time


cognitivemodeling

d a so o e pe ce e ed a so o e pe ce e e

Implementation of time Implementation of time estimationestimation

Idea: metronome that starts ticking fast but gradually Idea: metronome that starts ticking fast but gradually slows downslows downThe current value of the accumulator is available toThe current value of the accumulator is available toThe current value of the accumulator is available to The current value of the accumulator is available to the rest of cognitionthe rest of cognitionAt any moment, the accumulator can be read or At any moment, the accumulator can be read or

t h d i th diti f d ti lt h d i th diti f d ti lmatched in the condition of a production rulematched in the condition of a production rule


cognitivemodeling

Example: estimate 6 secondsExample: estimate 6 secondsReal timeReal time

Pulses

8 pulsesR dReproduce

“ i d !”“six seconds!”


cognitivemodeling

The module can fit basic time The module can fit basic time interval estimation propertiesinterval estimation properties

Scalar propertyData are from Rakitin et al (1998)Data are from Rakitin et al (1998)


cognitivemodeling

BisectionData are from Penney, Gibbon & Meck (2000)Data are from Penney, Gibbon & Meck (2000)

Data are from Rakitin et al. (1998)Data are from Rakitin et al. (1998)

DualDual--task Timing Tasktask Timing Task

Subjects have to estimate an interval of Subjects have to estimate an interval of unknown duration, but this is only one of unknown duration, but this is only one of , y, ythree tasks they have to dothree tasks they have to doThe difficulty of the other tasks isThe difficulty of the other tasks isThe difficulty of the other tasks is The difficulty of the other tasks is manipulated to see the impact on timing manipulated to see the impact on timing accuracyaccuracyaccuracyaccuracy


cognitivemodeling


cognitivemodeling

end of high profit score0

The start of a trialis marked by the

“

Ascore

0

Stimuli appear in both boxes

0

0 sec

appearance of “end of high profit” in the left box

0

0.3 sec

B

Stimuli keepA

score30

0.5 sec

Clicking with the mouse on targets yields 30 points

B

score300

A

B

13 sec

Stimuli keep appearing in both boxes until the 13 seconds of the trial have passed.

HIGH

Bscore

80

If the test button is clicked before 7 seconds have passed, nothing happens

Ascore200

A

When the stimulus in the left box is a target, the space key has to be

HIGH

A After 7 seconds the

6 sec

nothing happens, except for a deduction of 10 points

If the test button is clicked now stimuli

HIGH

[space key is pressed] 7.7 sec

key has to be pressed to score 100 points

score110

7 sec

high profit period starts, but this is not visible in the interface

Bscore100

A

7.5 sec

clicked now, stimuli in the left box or worth 100 points (and 10 points are deducted)


cognitivemodeling

Four conditionsFour conditions

Blocks consist of 5 trials of 120 secondsBlocks consist of 5 trials of 120 seconds•• LL: 4 blocks of lettersLL: 4 blocks of letters•• AA: 4 blocks of additionsAA: 4 blocks of additions•• LA: 2 blocks of letters, then 2 blocks with LA: 2 blocks of letters, then 2 blocks with

additionsadditions•• AL: 2 blocks of additions, then 2 blocks with AL: 2 blocks of additions, then 2 blocks with

l ttl ttlettersletters10 Subjects/condition10 Subjects/condition


cognitivemodeling

Bottom lineBottom line

Three tasks:Three tasks:•• Respond leftRespond leftRespond leftRespond left•• Respond rightRespond right•• Estimate an initially unknown time intervalEstimate an initially unknown time intervalEstimate an initially unknown time intervalEstimate an initially unknown time interval


cognitivemodeling

Errors in time estimationErrors in time estimation

Noise in the temporal moduleNoise in the temporal moduleModel was attending visual stimuli atModel was attending visual stimuli atModel was attending visual stimuli at Model was attending visual stimuli at the time it should have pressedthe time it should have pressedModel fails to accumulate enoughModel fails to accumulate enoughModel fails to accumulate enough Model fails to accumulate enough instances and therefore has to guess all instances and therefore has to guess all the timethe timethe timethe time


cognitivemodeling

Effect of difficulty of the Effect of difficulty of the secondary taskssecondary tasks

Attentional gate theory: when the Attentional gate theory: when the secondary task is more difficult, secondary task is more difficult, y ,y ,accumulation of pulses is sloweraccumulation of pulses is slowerACTACT--R model: when the secondary taskR model: when the secondary taskACTACT R model: when the secondary task R model: when the secondary task is more difficult, the probability that you is more difficult, the probability that you fail to pay attention to the timing task isfail to pay attention to the timing task isfail to pay attention to the timing task is fail to pay attention to the timing task is higherhigher


cognitivemodeling

Possible clashes:Possible clashes:““temporal reference memorytemporal reference memory”” is ordinary memory that is ordinary memory that we also need for other things (retrieving addition facts)we also need for other things (retrieving addition facts)Manual module is maybe in use to click something or Manual module is maybe in use to click something or a ua odu e s aybe use to c c so et g oa ua odu e s aybe use to c c so et g opush a keypush a key

M t hiss

Declarative Module


Matching

Selection

ExecutionProd

uctio

nsPr

oduc

tions

Problem BufferProblem BufferPacemaker Gate

StartSignal



E t l W ld

Manual BufferVisual BufferSignal


cognitivemodeling

External World

From: From: [email protected]@cmu.eduSubject: Prediction for dualSubject: Prediction for dual--task timing experimenttask timing experimentDate:Date: 7 March 2005 16:45:21 GMT+01:007 March 2005 16:45:21 GMT+01:00Date: Date: 7 March 2005 16:45:21 GMT+01:007 March 2005 16:45:21 GMT+01:00To: To: [email protected]@andrew.cmu.eduOne of the goals of cognitive modeling is to make predictions One of the goals of cognitive modeling is to make predictions instead of "postdictions" In order to stay true to this goal I haveinstead of "postdictions" In order to stay true to this goal I haveinstead of "postdictions". In order to stay true to this goal, I have instead of "postdictions". In order to stay true to this goal, I have posted a prediction for an experiment we are about to start on the posted a prediction for an experiment we are about to start on the web. You can find it (and the details of the experiment) web. You can find it (and the details of the experiment) on:on:http://www ai rug nl/~niels/prediction htmlhttp://www ai rug nl/~niels/prediction htmlon:on:http://www.ai.rug.nl/ niels/prediction.htmlhttp://www.ai.rug.nl/ niels/prediction.html======================================================================================================


cognitivemodeling

Predictions ACTPredictions ACT--R vs. R vs. Attentional Gate TheoryAttentional Gate Theory


cognitivemodeling

Predictions ACTPredictions ACT--R vs. R vs. Attentional Gate TheoryAttentional Gate Theory

Easy Easy to to HardHard

HardHardtotototoEasyEasy


cognitivemodeling

Hard vs. EasyHard vs. EasyData Model


cognitivemodeling

Data Model


cognitivemodeling

Failing to attend the time at allFailing to attend the time at all

0.8

0.9

1

0.8

0.9

1

Hard to Hard Hard to Easy

0 4

0.5

0.6

0.7

rcen

tag

e m

isse

d

LL pred

AA pred

LL data

AA data0 4

0.5

0.6

0.7

rcen

tag

e m

isse

d

LA pred

AL pred

LA data

AL data

0.1

0.2

0.3

0.4

Per

0.1

0.2

0.3

0.4

Per

E t EEasy to Hard

0

Block 1 Block 2 Block 3 Block 4

0

Block 1 Block 2 Block 3 Block 4Easy to Easy


cognitivemodeling

ConcludingConcluding

It is hard to completely rule out a timingIt is hard to completely rule out a timing--specific attentional componentspecific attentional componentp pp pBut we should try to explain effects of But we should try to explain effects of attention as much as possible byattention as much as possible byattention as much as possible by attention as much as possible by general principles of attentiongeneral principles of attention


cognitivemodeling

Can people add and subtract Can people add and subtract time intervals?time intervals?

People (and animals) seem to be able People (and animals) seem to be able to handle multiple overlapping time to handle multiple overlapping time p pp gp pp gintervalsintervalsBut do they use separate timers forBut do they use separate timers forBut do they use separate timers for But do they use separate timers for these intervals, or do they uses a single these intervals, or do they uses a single timer to track multiple intervalstimer to track multiple intervalstimer to track multiple intervalstimer to track multiple intervals

van Rijn, H. & Taatgen, N.A. (2008). Timing of multiple overlapping time intervals: How l k d h ? A t P h l i 129(3) 365 375


cognitivemodeling

many clocks do we have? Acta Psychologica, 129(3), 365-375.

Dual timing paradigmDual timing paradigmInterval =2 secCorrect =1.75-2.25 sec

Here yout t t

Here yout d

Here youk

Here youkget a start

signalget a secondstart signal

press a keywith left finger

press a keywith right finger


cognitivemodeling


cognitivemodeling

Onset time can be longerOnset time can be longer


cognitivemodeling

ModelModel

Start theclock

Respondwhen timerreaches A

Store Respondwhen timerthe timer

(C pulses)

when timerreaches A+C


cognitivemodeling

NonNon--linear scale biases linear scale biases second estimatesecond estimate


cognitivemodeling

Result: Estimate of second Result: Estimate of second interval as a function of SOAinterval as a function of SOA


cognitivemodeling

Memory for time intervalsMemory for time intervals

If you let people learn to intervals, how If you let people learn to intervals, how do they affect each other?do they affect each other?yyExperiment: people have to alternate in Experiment: people have to alternate in producing a 2 second and a 3 1 secondproducing a 2 second and a 3 1 secondproducing a 2 second and a 3.1 second producing a 2 second and a 3.1 second interval. They receive feedback (too interval. They receive feedback (too long too short correct)long too short correct)long, too short, correct)long, too short, correct)

Taatgen, N.A. & van Rijn, H. (submitted). Trace of times past:


cognitivemodeling

g j ( ) prepresentations of temporal intervals in memory.

ResultsResultsEstimates for 2 second interval are slightly longer andslightly longer, and the estimates for the 3.1 second interval are shorter (despiteare shorter (despite feedback)

Question: do they really affect each other?


cognitivemodeling

other?

New experiment, similar setupNew experiment, similar setup

Manipulation: the long (3.1 second) Manipulation: the long (3.1 second) interval changes during the experimentinterval changes during the experimentg g pg g pThe short interval stays the sameThe short interval stays the same


cognitivemodeling

ResultsResults


cognitivemodeling

Analysis: MixedAnalysis: Mixed--Effect modelsEffect modelsStart Start withwith a a simplesimple regressionregression equationequation::

shortshortn sn s == ββ00 ++ rrss ++ εεn sn sshortshortn,sn,s ββ00 rrss εεn,sn,s

ThenThen addadd factors as long as the more factors as long as the more ggcomplex complex equationequation fits the data fits the data significantlysignificantlybetterbetter thanthan the the previousprevious modelmodel

shortshortn,sn,s = = ββ00 + + ββ11shortshortnn--1,s1,s ++ ββ22shortshortnn--2,s2,s + + rrss + + εεn,sn,s


cognitivemodeling

Final set of factors for short Final set of factors for short intervalinterval

Fixed Effect Value of β t valueIntercept 657 ms 4.6hshortn-1 0.385 8.3

shortn-2 0.085 3.3short fb S 110 ms 3 1short-fb-Sn-1 110 ms 3.1short-fb-Ln-1 -208 ms -6.5longn 1 0.16 5.1gn-1long-fb-Sn-1 92.6 ms 3.2long-fb-Ln-1 -163 ms -4.2


cognitivemodeling

How to explain that? How to explain that? A memory modelA memory model

Assumption: we collect experiences in a Assumption: we collect experiences in a memory memory ““poolpool””yy ppWhenever we need to determine how Whenever we need to determine how long to wait for the next interval we dolong to wait for the next interval we dolong to wait for the next interval, we do long to wait for the next interval, we do a weighted average over the elements a weighted average over the elements in the memory poolin the memory poolin the memory poolin the memory pool


cognitivemodeling

Weighted average dependsMemory PoolMemory Pool average depends on:- How long ago

Short

LongP = 24t = -16

was the experience

- Does the P = 17t = -20

t 16

Long

experience match the query?

ShortP = 16

LongP = 25t = -8

query?

t = -11 ShortP = ??t = 0


cognitivemodeling

t 0

Declarative memory with Declarative memory with blendingblending

Model uses baseModel uses base--level leaning and a level leaning and a mismatch penalty when on a short/long mismatch penalty when on a short/long p y gp y gmismatch:mismatch:


cognitivemodeling

BlendingBlending

EachEach chunkchunk has a has a probabilityprobability of of beingbeingretrievedretrieved::

The The retrievedretrieved interval is a interval is a blendblend of of allallh kh kchunkschunks


cognitivemodeling

HowHow to calculate the blend?to calculate the blend?

Calculate the Calculate the activations of all activations of all

ExampleExample: : retrieveretrieve a a blendblend forfor long:long:

B(short, 9)=2.0B(short, 9)=2.0candidate chunkscandidate chunksApply mismatch Apply mismatch

lti t h klti t h k

( , )( , )B(short,10)=1.0B(short,10)=1.0B(long,16)=2.0B(long,16)=2.0B(long,17)=1.5B(long,17)=1.5penalties to chunks penalties to chunks

that do not match that do not match completelycompletely

B(long,17) 1.5B(long,17) 1.5ApplyApply penalty (of penalty (of --2):2):

A(short, 9)=0.0A(short, 9)=0.0A(short 10)=A(short 10)= 1 01 0completelycompletely A(short,10)=A(short,10)=--1.01.0A(long,16)=2.0A(long,16)=2.0A(long,17)=1.5A(long,17)=1.5


cognitivemodeling

How to calculate the blend?How to calculate the blend?

NowNow calculatecalculate the the probabilityprobability of of recallrecall

ApplyApply penalty:penalty:A(short, 9)=0.0A(short, 9)=0.0A(short 10)=A(short 10)= 1 01 0forfor eacheach of the of the

candidatescandidates, , usingusing::

A(short,10)=A(short,10)=--1.01.0A(long,16)=2.0A(long,16)=2.0A(long,17)=1.5A(long,17)=1.5

ResultsResults in (t=1):in (t=1):p(short, 9)=0.076p(short, 9)=0.076p(short 10)=0 0278p(short 10)=0 0278p(short, 10)=0.0278p(short, 10)=0.0278p(long, 16)=0.558p(long, 16)=0.558p(long, 17)=0.339p(long, 17)=0.339


cognitivemodeling

How to calculate the blend?How to calculate the blend?

MultiplyMultiply eacheachprobabilityprobability withwith the the

ProbabilitiesProbabilitiesp(short, 9)=0.076p(short, 9)=0.076( h t( h tslot slot valuevalue, , andand addadd itit

allall up up toto get the get the blendedblended valuevalue

p(short, p(short, 10)=0.027810)=0.0278p(long, 16)=0.558p(long, 16)=0.558blendedblended valuevalue p( g, )p( g, )p(long, 17)=0.339p(long, 17)=0.339

0.076*9 + 0.0278*10 0.076*9 + 0.0278*10 0 558* 16 0 3390 558* 16 0 339+ 0.558* 16 + 0.339 + 0.558* 16 + 0.339

* 17 =* 17 = 15.6515.65


cognitivemodeling

Handling feedbackHandling feedbackIn order to handle feedback, the model In order to handle feedback, the model also stores the feedback on the previous also stores the feedback on the previous trial in declarative memorytrial in declarative memorytrial in declarative memorytrial in declarative memory““Too shortToo short”” is stored as a positive number, is stored as a positive number, ““Too longToo long”” as a negative number, and as a negative number, and gg gg““CorrectCorrect”” as 0.as 0.To determine the number of pulses to wait, To determine the number of pulses to wait, the model retrieves the duration and thethe model retrieves the duration and thethe model retrieves the duration and the the model retrieves the duration and the feedback from memory, and adds these feedback from memory, and adds these togethertogether


cognitivemodeling

This model fits the data quite This model fits the data quite nicelynicely


cognitivemodeling

We can run the same regression model We can run the same regression model e ca u e sa e eg ess o odee ca u e sa e eg ess o odeon the model outputon the model output

Fixed Effect β data β Model Intercept 657 ms 789 msshortn-1 0.385 0.356shortn-2 0.085 0.048h t fb S 110 170short-fb-Sn-1 110 ms 170 ms

short-fb-Ln-1 -208 ms -153 mslong 1 0.16 0.15longn-1 0.16 0.15long-fb-Sn-1 92.6 ms 125 mslong-fb-Ln-1 -163 ms -211 ms


cognitivemodeling

ConclusionsConclusions

Architecture approach offers great Architecture approach offers great advantages:advantages:gg•• Allows predictionAllows prediction•• Allows integration of time perception withAllows integration of time perception withAllows integration of time perception with Allows integration of time perception with

other aspects of cognitionother aspects of cognition•• Allows us to use general theories of Allows us to use general theories of gg

attention and memory instead of timeattention and memory instead of time--estimation specific theoriesestimation specific theories


cognitivemodeling

Unconscious time perceptionUnconscious time perception

ChoiceChoice--reaction time reaction time experimentexperimentppThe interThe inter--stimulus interval stimulus interval was varied on the last trialwas varied on the last trialwas varied on the last trial was varied on the last trial of each blockof each block

Grosjean et al., 2001


cognitivemodeling

TimeTime--line of a blockline of a blockSti lStimulus appears

Participantdecides and

k

Inter-stimulusinterval

presses key


cognitivemodeling

DesignDesignLengthening:ISI = 350mslast = 467ms

Control:ISI = 467mslast = 467ms467ms

Shortening:ISI = 700mslast = 467ms


cognitivemodeling

DataData


cognitivemodeling

ModelModel

What could be the profit of timing?What could be the profit of timing?If we know when and where the information If we know when and where the information will be, we can immediately do a +visual> on will be, we can immediately do a +visual> on the location without waiting for the =visualthe location without waiting for the =visual--l ti i t 50l ti i t 50location>, saving up to 50mslocation>, saving up to 50msBut if the stimulus is early, we cannot use this But if the stimulus is early, we cannot use this

d t b it illd t b it ill ““ ii ””advantage, because it will advantage, because it will ““surprisesurprise”” usus


cognitivemodeling

At the start of the trial we start At the start of the trial we start the timerthe timer

(p start-timer=goal>isa crtstatus nil>==>

+temporal>isa timeisa time

=goal>status waiting)


cognitivemodeling

g)

When the stimulus comes up, we When the stimulus comes up, we process it, but also store the timeprocess it, but also store the time

(p found-visual-location(p=goal>isa crtstatus waiting

=visual-location>isa visual-location

=visual-state>isa module-statemodality free

=temporal>isa timeticks =ticks

==>i l>+visual>

isa visual-objectscreen-pos =visual-location

=goal>time =ticks


cognitivemodeling

time =ticksloc =visual-locationstatus wait-visual

For the next trial, we can try to anticipate For the next trial, we can try to anticipate o e e a , e ca y o a c pa eo e e a , e ca y o a c pa ethe stimulus, and save up to 50msthe stimulus, and save up to 50ms

(p expect-visual-location=goal>isa crtstatus waitingstatus waitingtime =ticksloc =loc

=temporal>isa timeisa timeticks =ticks

==>+visual>i i l bj tisa visual-objectscreen-pos =loc

=goal>status wait-visual)


cognitivemodeling

How does the model save How does the model save time?time?

Real time Stimulus50ms

Real time

Ticks

Found-visual-

location

8Ne t trialNext trial

8 Expect-visual-

location

8 Expect-visual-

location


cognitivemodeling

Found-visual-

location7

Results of the model with no Results of the model with no parameter fittingparameter fitting


cognitivemodeling

perception of time -...

Documents