![Page 1: Some Markov models for direct observation of behavior · Direct observation of behavior •Quantities of interest •Prevalence: proportion of time that a behavior occurs •Incidence:](https://reader036.vdocuments.site/reader036/viewer/2022081607/5eddbb0dad6a402d6668e7e9/html5/thumbnails/1.jpg)
Some Markov models for direct observation of behavior
James E. Pustejovsky
Northwestern University
May 29, 2013
![Page 2: Some Markov models for direct observation of behavior · Direct observation of behavior •Quantities of interest •Prevalence: proportion of time that a behavior occurs •Incidence:](https://reader036.vdocuments.site/reader036/viewer/2022081607/5eddbb0dad6a402d6668e7e9/html5/thumbnails/2.jpg)
Direct observation of behavior
• Quantities of interest • Prevalence: proportion of time that a behavior occurs
• Incidence: rate at which new behavioral events begin
• Intensity, contingency, others
• Applications in psychology and education research • Measurement of teaching practice
• Measurement of student behavior
• Evaluating interventions for individuals with disabilities
• Other examples in animal behavior, organizational psychology, social work, exercise physiology
2
![Page 3: Some Markov models for direct observation of behavior · Direct observation of behavior •Quantities of interest •Prevalence: proportion of time that a behavior occurs •Incidence:](https://reader036.vdocuments.site/reader036/viewer/2022081607/5eddbb0dad6a402d6668e7e9/html5/thumbnails/3.jpg)
Observation recording methods
• How to turn direct observation of a “behavior stream” into data?
• Continuous recording methods • Produce rich data, amenable to sophisticated modeling
• Effort-intensive
• Discontinuous recording methods • Less demanding methods needed in field settings
• Momentary time sampling
• Interval recording
• Other possibilities?
3
![Page 4: Some Markov models for direct observation of behavior · Direct observation of behavior •Quantities of interest •Prevalence: proportion of time that a behavior occurs •Incidence:](https://reader036.vdocuments.site/reader036/viewer/2022081607/5eddbb0dad6a402d6668e7e9/html5/thumbnails/4.jpg)
Outline
• Model for behavior stream as observed
• Momentary time sampling
• Interval recording
• Some novel proposals
• Efficiency considerations
4
![Page 5: Some Markov models for direct observation of behavior · Direct observation of behavior •Quantities of interest •Prevalence: proportion of time that a behavior occurs •Incidence:](https://reader036.vdocuments.site/reader036/viewer/2022081607/5eddbb0dad6a402d6668e7e9/html5/thumbnails/5.jpg)
A model for the behavior stream
5
Session time
Interim times
Event durations
D1 D2 D3
E0 E1 E2 E3
1
1 0
is not occuring at time
is occu
0 behavior( )
ring at tim1 beh e avior
0j
i i j
j i
t
tY t
I Dt E D
Behavior stream MTS Interval recording Novel proposals Efficiency
![Page 6: Some Markov models for direct observation of behavior · Direct observation of behavior •Quantities of interest •Prevalence: proportion of time that a behavior occurs •Incidence:](https://reader036.vdocuments.site/reader036/viewer/2022081607/5eddbb0dad6a402d6668e7e9/html5/thumbnails/6.jpg)
Assumptions
1. Event durations:
2. Interim times:
3. Event durations and interim times are all mutually independent.
4. Process is in equilibrium.
Under this model:
• Prevalence ϕ = μ / (μ + λ)
• Incidence ζ = 1 / (μ + λ)
Alternating Poisson Process
6
Exp(1/ ), 1, 2, , ..~ 3 .jD j
Exp(1/ ), 1, 2, , ..~ 3 .jE j
Behavior stream MTS Interval recording Novel proposals Efficiency
![Page 7: Some Markov models for direct observation of behavior · Direct observation of behavior •Quantities of interest •Prevalence: proportion of time that a behavior occurs •Incidence:](https://reader036.vdocuments.site/reader036/viewer/2022081607/5eddbb0dad6a402d6668e7e9/html5/thumbnails/7.jpg)
Behavior stream MTS Interval recording Novel proposals Efficiency
Momentary time sampling (MTS)
7
• (K + 1) moments, equally spaced at intervals of length L.
• Observer records the presence or absence of a behavior at each moment
• Recorded data are
( ), 0,...,k Y kL k KX
X0 = 0 X1 = 1 X2 = 0 X3 = 0 X4 = 1 X5 = 1 X6 = 1 X7 = 1 X8 = 0
Session time
![Page 8: Some Markov models for direct observation of behavior · Direct observation of behavior •Quantities of interest •Prevalence: proportion of time that a behavior occurs •Incidence:](https://reader036.vdocuments.site/reader036/viewer/2022081607/5eddbb0dad6a402d6668e7e9/html5/thumbnails/8.jpg)
Model for MTS data • Under the alternating Poisson process, X1,…,XK follow a
discrete-time Markov chain (DTMC) with two states (see e.g., Kulkarni, 2010).
8
X=0 X=1
p0(L) 1 - p0(L)
p1(L) 1 - p1(L)
0
1
( ) Pr( ( ) 1| (0) 0) 1 exp(1 )
( ) Pr( ( ) 1| (0) 1) (1 )exp(1 )
tt Y t Y
tp t
p
Y t Y
Behavior stream MTS Interval recording Novel proposals Efficiency
![Page 9: Some Markov models for direct observation of behavior · Direct observation of behavior •Quantities of interest •Prevalence: proportion of time that a behavior occurs •Incidence:](https://reader036.vdocuments.site/reader036/viewer/2022081607/5eddbb0dad6a402d6668e7e9/html5/thumbnails/9.jpg)
MTS model, continued
• Maximum likelihood estimators of ϕ and ζ have closed form expressions (Brown, Solomon, & Stephens, 1975).
• But under more general models.
• Extensive literature, lots of generalizations • stopping rules for observation time
(Brown, Solomon, & Stephens, 1977, 1979; Griffin & Adams, 1983)
• Irregular observation times (e.g., Cook, 1999)
• Random effects to describe variation across subjects (e.g., Cook et al., 1999)
9
E X
Behavior stream MTS Interval recording Novel proposals Efficiency
![Page 10: Some Markov models for direct observation of behavior · Direct observation of behavior •Quantities of interest •Prevalence: proportion of time that a behavior occurs •Incidence:](https://reader036.vdocuments.site/reader036/viewer/2022081607/5eddbb0dad6a402d6668e7e9/html5/thumbnails/10.jpg)
Partial interval recording (PIR)
10
• Divide period into K intervals, each of length L.
• For each interval, observer records whether behavior occurred at any point during the interval.
• Recorded data are
U1 = 1 U2 = 1 U3 = 0 U4 = 1 U5 = 1 U6 = 1 U7 = 1 U8 = 1
Session time
[0, )
0 1 , 1,..., .L
k IU Y k L t dt k K
Behavior stream MTS Interval recording Novel proposals Efficiency
![Page 11: Some Markov models for direct observation of behavior · Direct observation of behavior •Quantities of interest •Prevalence: proportion of time that a behavior occurs •Incidence:](https://reader036.vdocuments.site/reader036/viewer/2022081607/5eddbb0dad6a402d6668e7e9/html5/thumbnails/11.jpg)
PIR, continued • Unlike MTS, the mean of PIR data is not readily
interpretable:
11
(1 ) 1 exp1
LE U
Behavior stream MTS Interval recording Novel proposals Efficiency
![Page 12: Some Markov models for direct observation of behavior · Direct observation of behavior •Quantities of interest •Prevalence: proportion of time that a behavior occurs •Incidence:](https://reader036.vdocuments.site/reader036/viewer/2022081607/5eddbb0dad6a402d6668e7e9/html5/thumbnails/12.jpg)
Model for PIR data • Define Vk as the number of consecutive intervals where
behavior is present:
• Under the alternating Poisson process, V1,…,VK follow a DTMC on the space {0,1,2,3,…}.
12
max 0 : U 0 .k jk j kV
V=0 V=1
π01 1 - π01
V=2 V=3 …
1 – π12 1 – π23 1 – π34
π12 π23 π34
Behavior stream MTS Interval recording Novel proposals Efficiency
![Page 13: Some Markov models for direct observation of behavior · Direct observation of behavior •Quantities of interest •Prevalence: proportion of time that a behavior occurs •Incidence:](https://reader036.vdocuments.site/reader036/viewer/2022081607/5eddbb0dad6a402d6668e7e9/html5/thumbnails/13.jpg)
Whole interval recording (WIR)
13
• Divide period into K intervals, each of length L.
• For each interval, observer records whether behavior occurred for the duration of the interval.
• Recorded data are
• Equivalent to PIR for absence of event.
W1 = 0 W2 = 0 W3 = 0 W4 = 0 W5 = 1 W6 = 1 W7 = 0 W8 = 0
Session time
[0, )
1 , 1,..., .L
k IW L Y k L t dt k K
Behavior stream MTS Interval recording Novel proposals Efficiency
![Page 14: Some Markov models for direct observation of behavior · Direct observation of behavior •Quantities of interest •Prevalence: proportion of time that a behavior occurs •Incidence:](https://reader036.vdocuments.site/reader036/viewer/2022081607/5eddbb0dad6a402d6668e7e9/html5/thumbnails/14.jpg)
Augmented interval recording (AIR)
14
• Divide period into K/2 intervals, each of length 2L.
• Use MTS at the beginning of each interval, to record Xk-1.
• If Xk-1 = 0, use PIR for the remainder of the interval.
• If Xk-1 = 1, use WIR for the remainder of the interval.
X0 = 0 X1 = 0 X2 = 1 X3 = 1 X4 = 0
U1 = 1 U2 = 1 W3 = 1 W4 = 0
Session time
Behavior stream MTS Interval recording Novel proposals Efficiency
![Page 15: Some Markov models for direct observation of behavior · Direct observation of behavior •Quantities of interest •Prevalence: proportion of time that a behavior occurs •Incidence:](https://reader036.vdocuments.site/reader036/viewer/2022081607/5eddbb0dad6a402d6668e7e9/html5/thumbnails/15.jpg)
Model for AIR data
15
• Define Zk = Uk + Wk + Xk .
• Under the alternating Poisson process, Z1,…,ZK/2 follow a DTMC on {0,1,2,3}, with transition probabilities πab= Pr(Zk = b | Xk-1 = a)
Z=0 U=0, W=0, X=0
Z=1 U=1, W=0, X=0
Z=2 U=1, W=0, X=1
Z=3 U=1, W=1, X=1
π01
π00
π00
π01
π02
π11
π11
π02
π12
π13
π12
π13
Behavior stream MTS Interval recording Novel proposals Efficiency
![Page 16: Some Markov models for direct observation of behavior · Direct observation of behavior •Quantities of interest •Prevalence: proportion of time that a behavior occurs •Incidence:](https://reader036.vdocuments.site/reader036/viewer/2022081607/5eddbb0dad6a402d6668e7e9/html5/thumbnails/16.jpg)
Intermittent transition recording (ITR)
16
• Divide period into K/2 intervals, each of length 2L.
• Use MTS at the beginning of each interval, to record Xk-1.
• Record time until next transition as Tk.
X0 = 0 X1 = 0 X2 = 1 X3 = 1 X4 = 0
T1 T2 T3 >2L T4
Session time
Behavior stream MTS Interval recording Novel proposals Efficiency
![Page 17: Some Markov models for direct observation of behavior · Direct observation of behavior •Quantities of interest •Prevalence: proportion of time that a behavior occurs •Incidence:](https://reader036.vdocuments.site/reader036/viewer/2022081607/5eddbb0dad6a402d6668e7e9/html5/thumbnails/17.jpg)
• Under the alternating Poisson process, T1,X1, …,TK/2,XK/2 have the property that
F(Tk,Xk | T1,X1,…,Tk-1,Xk-1) = F(Tk,Xk|Xk-1)
Model for ITR data
17 Behavior stream MTS Interval recording Novel proposals Efficiency
X=0 X=1
p0(L-t) 1 - p0(L-t)
p1(L-t) 1 - p1(L-t)
T | X=1
T | X=0
fμ(t)
fλ(t)
![Page 18: Some Markov models for direct observation of behavior · Direct observation of behavior •Quantities of interest •Prevalence: proportion of time that a behavior occurs •Incidence:](https://reader036.vdocuments.site/reader036/viewer/2022081607/5eddbb0dad6a402d6668e7e9/html5/thumbnails/18.jpg)
Asymptotic relative efficiency
18 Behavior stream MTS Interval recording Novel proposals Efficiency
• Procedure p, q ∈ {MTS, PIR, AIR, ITR}
• are maximum likelihood estimators based on procedure p
• are asymptotic variances based on inverse of expected information matrix.
• Asymptotic relative efficiency of p versus q
ˆVˆ ˆ,R
ˆE
VA
q
p q
p
ˆVˆ ˆ,R
ˆE
VA
q
p q
p
ˆ ˆ,p p
,ˆ ˆp pV V
![Page 19: Some Markov models for direct observation of behavior · Direct observation of behavior •Quantities of interest •Prevalence: proportion of time that a behavior occurs •Incidence:](https://reader036.vdocuments.site/reader036/viewer/2022081607/5eddbb0dad6a402d6668e7e9/html5/thumbnails/19.jpg)
PIR AIR ITR
0.0
0.2
0.4
0.6
0.8
0.0
0.2
0.4
0.6
0.8
0.0
0.2
0.4
0.6
0.8
MT
SP
IRA
IR
0.0 0.2 0.4 0.6 0.8 0.0 0.2 0.4 0.6 0.8 0.0 0.2 0.4 0.6 0.8
True prevalence
Tru
e in
cid
en
ce
(fr
eq
ue
ncy p
er
inte
rva
l)
0.5
1.0
2.0
asymptotic relative efficiency
Asymptotic relative efficiency: Prevalence
19 Behavior stream MTS Interval recording Novel proposals Efficiency
PIR AIR ITR
0.0
0.2
0.4
0.6
0.8
0.0
0.2
0.4
0.6
0.8
0.0
0.2
0.4
0.6
0.8
MT
SP
IRA
IR
0.0 0.2 0.4 0.6 0.8 0.0 0.2 0.4 0.6 0.8 0.0 0.2 0.4 0.6 0.8
True prevalence
Tru
e in
cid
en
ce
(fr
eq
ue
ncy p
er
inte
rva
l)
0.95
1.00
1.05
asymptotic relative efficiency
blue = column is more efficient red = row is more efficient
![Page 20: Some Markov models for direct observation of behavior · Direct observation of behavior •Quantities of interest •Prevalence: proportion of time that a behavior occurs •Incidence:](https://reader036.vdocuments.site/reader036/viewer/2022081607/5eddbb0dad6a402d6668e7e9/html5/thumbnails/20.jpg)
PIR AIR ITR
0.0
0.2
0.4
0.6
0.8
0.0
0.2
0.4
0.6
0.8
0.0
0.2
0.4
0.6
0.8
MT
SP
IRA
IR
0.0 0.2 0.4 0.6 0.8 0.0 0.2 0.4 0.6 0.8 0.0 0.2 0.4 0.6 0.8
True prevalence
Tru
e in
cid
en
ce
(fr
eq
ue
ncy p
er
inte
rva
l)
0.5
1.0
2.0
asymptotic relative efficiency
Asymptotic relative efficiency: Incidence
20 Behavior stream MTS Interval recording Novel proposals Efficiency
blue = column is more efficient red = row is more efficient
![Page 21: Some Markov models for direct observation of behavior · Direct observation of behavior •Quantities of interest •Prevalence: proportion of time that a behavior occurs •Incidence:](https://reader036.vdocuments.site/reader036/viewer/2022081607/5eddbb0dad6a402d6668e7e9/html5/thumbnails/21.jpg)
• Evaluating these models & methods • Field testing
• When is it okay to treat ML estimates from individual sessions as “pre-processing”?
• Lots still to do • Build data-collection software
• Extensions to between-period regression models
• Random period/subject effects
• PIR, AIR, ITR under other distributional assumptions?
Future work
21
![Page 23: Some Markov models for direct observation of behavior · Direct observation of behavior •Quantities of interest •Prevalence: proportion of time that a behavior occurs •Incidence:](https://reader036.vdocuments.site/reader036/viewer/2022081607/5eddbb0dad6a402d6668e7e9/html5/thumbnails/23.jpg)
PIR model • Transition probabilities are uglier than MTS:
where and f(j) is the j-fold recursion of f.
23
/ )
,
(
1 1 1 (Pr 1| 0)1 L
j j k k
jV j V j fe
/ (1 )
/(1 )( )
1
( )
(1 )
L
L
qf q
q e
e
![Page 24: Some Markov models for direct observation of behavior · Direct observation of behavior •Quantities of interest •Prevalence: proportion of time that a behavior occurs •Incidence:](https://reader036.vdocuments.site/reader036/viewer/2022081607/5eddbb0dad6a402d6668e7e9/html5/thumbnails/24.jpg)
AIR, continued
24
Event occurring at time (k-1)L?
Event ends before time kL?
Event begins before time kL?
Xk-1=1, Uk=1
Xk-1=0, Wk=0
Yes
No
Wk=1
Wk=0
Yes
No
Vk=1
Vk=0
Yes
No
Behavior stream MTS Interval recording Novel proposals Efficiency
![Page 25: Some Markov models for direct observation of behavior · Direct observation of behavior •Quantities of interest •Prevalence: proportion of time that a behavior occurs •Incidence:](https://reader036.vdocuments.site/reader036/viewer/2022081607/5eddbb0dad6a402d6668e7e9/html5/thumbnails/25.jpg)
AIR model
25 Extras
• Transition probabilities are given by
2 /
00
10 0
Le
2 /
01 0
11 0
(2 )
( )
1
2
L L
p L
e p
02 0
2 /
12 01
(2 )
(2 )L
p
e p
L
L
03
2 /
13
0
Le
![Page 26: Some Markov models for direct observation of behavior · Direct observation of behavior •Quantities of interest •Prevalence: proportion of time that a behavior occurs •Incidence:](https://reader036.vdocuments.site/reader036/viewer/2022081607/5eddbb0dad6a402d6668e7e9/html5/thumbnails/26.jpg)
• Under alternating Poisson process,
ITR model
26
1 1 /2 /2 0
/2
1
1
1
, ,..., , |
; , ,;| , |
K K
K
k k k
k
k k
f t x
f t x
x t x
x f x t
Behavior stream MTS Interval recording Novel proposals Efficiency