tracking of animals using airborne...
TRANSCRIPT
Tracking of AnimalsUsing Airborne CamerasClas Veibäck
1 Introduction2 Background3 Constrained Motion Model4 Uncertain Timestamp Model5 Mode Observations6 Conclusions
LINK-SIC Presentation Clas Veibäck November 7, 2016 2
Introduction
• Tracking of animals
• Overhead cameras
• Applicable to other target
types
• Applications to demonstrate
theory
LINK-SIC Presentation Clas Veibäck November 7, 2016 2
Introduction
• Tracking of animals
• Overhead cameras
• Applicable to other target
types
• Applications to demonstrate
theory
LINK-SIC Presentation Clas Veibäck November 7, 2016 2
Introduction
• Tracking of animals
• Overhead cameras
• Applicable to other target
types
• Applications to demonstrate
theory
LINK-SIC Presentation Clas Veibäck November 7, 2016 2
Introduction
• Tracking of animals
• Overhead cameras
• Applicable to other target
types
• Applications to demonstrate
theory
LINK-SIC Presentation Clas Veibäck November 7, 2016 3
Main Contributions
Uncertain Timestamps
Constrained Motion Model Mode Observations
M1
M2
M1
M2
M1
M2
k− 1 k k+ 1
LINK-SIC Presentation Clas Veibäck November 7, 2016 3
Main Contributions
Uncertain Timestamps
Constrained Motion Model
Mode Observations
M1
M2
M1
M2
M1
M2
k− 1 k k+ 1
LINK-SIC Presentation Clas Veibäck November 7, 2016 3
Main Contributions
Uncertain Timestamps
Constrained Motion Model Mode Observations
M1
M2
M1
M2
M1
M2
k− 1 k k+ 1
LINK-SIC Presentation Clas Veibäck November 7, 2016 4
Dolphin Application
• Dolphinarium at Kolmården
Wildlife Park
• Fisheye camera with
occlusions
• Reflections and changing
light conditions
• Constrained to basin
LINK-SIC Presentation Clas Veibäck November 7, 2016 4
Dolphin Application
• Dolphinarium at Kolmården
Wildlife Park
• Fisheye camera with
occlusions
• Reflections and changing
light conditions
• Constrained to basin
LINK-SIC Presentation Clas Veibäck November 7, 2016 4
Dolphin Application
• Dolphinarium at Kolmården
Wildlife Park
• Fisheye camera with
occlusions
• Reflections and changing
light conditions
• Constrained to basin
LINK-SIC Presentation Clas Veibäck November 7, 2016 4
Dolphin Application
• Dolphinarium at Kolmården
Wildlife Park
• Fisheye camera with
occlusions
• Reflections and changing
light conditions
• Constrained to basin
LINK-SIC Presentation Clas Veibäck November 7, 2016 5
Bird Application
• Recording of birds in Emlen
funnels
• Detect take-off times
• Estimate take-off directions
LINK-SIC Presentation Clas Veibäck November 7, 2016 5
Bird Application
• Recording of birds in Emlen
funnels
• Detect take-off times
• Estimate take-off directions
LINK-SIC Presentation Clas Veibäck November 7, 2016 5
Bird Application
• Recording of birds in Emlen
funnels
• Detect take-off times
• Estimate take-off directions
LINK-SIC Presentation Clas Veibäck November 7, 2016 6
Orienteering Application
• GPS trajectory
• Control position known
• Improve position estimate
LINK-SIC Presentation Clas Veibäck November 7, 2016 6
Orienteering Application
• GPS trajectory
• Control position known
• Improve position estimate
LINK-SIC Presentation Clas Veibäck November 7, 2016 6
Orienteering Application
• GPS trajectory
• Control position known
• Improve position estimate
1 Introduction2 Background3 Constrained Motion Model4 Uncertain Timestamp Model5 Mode Observations6 Conclusions
LINK-SIC Presentation Clas Veibäck November 7, 2016 8
Target Tracking
Tracker
Presentation
Pre-processing Association TrackManagement
Estimation
Sensor Data
LINK-SIC Presentation Clas Veibäck November 7, 2016 9
Target Model
Linear Gaussian state-space model
xk = Fkxk−1 +wk,
wk ∼ N (0,Qk),
yj = Hjxj + vj , vj ∼ N (0,Rj),
x0 ∼ N (x0,P0).
LINK-SIC Presentation Clas Veibäck November 7, 2016 9
Target Model
Linear Gaussian state-space model
xk = Fkxk−1 +wk, wk ∼ N (0,Qk),
yj = Hjxj + vj , vj ∼ N (0,Rj),
x0 ∼ N (x0,P0).
LINK-SIC Presentation Clas Veibäck November 7, 2016 9
Target Model
Linear Gaussian state-space model
xk = Fkxk−1 +wk, wk ∼ N (0,Qk),
yj = Hjxj + vj ,
vj ∼ N (0,Rj),
x0 ∼ N (x0,P0).
LINK-SIC Presentation Clas Veibäck November 7, 2016 9
Target Model
Linear Gaussian state-space model
xk = Fkxk−1 +wk, wk ∼ N (0,Qk),
yj = Hjxj + vj , vj ∼ N (0,Rj),
x0 ∼ N (x0,P0).
LINK-SIC Presentation Clas Veibäck November 7, 2016 9
Target Model
Linear Gaussian state-space model
xk = Fkxk−1 +wk, wk ∼ N (0,Qk),
yj = Hjxj + vj , vj ∼ N (0,Rj),
x0 ∼ N (x0,P0).
LINK-SIC Presentation Clas Veibäck November 7, 2016 10
Camera Model
z
yx
p
xr
xc
yx
f
• Camera extrinsics
• Camera intrinsics
• Projection
• Perspective compensation
• Lens distortion
LINK-SIC Presentation Clas Veibäck November 7, 2016 10
Camera Model
z
yx
p
xr
xc
yx
f
• Camera extrinsics
• Camera intrinsics
• Projection
• Perspective compensation
• Lens distortion
LINK-SIC Presentation Clas Veibäck November 7, 2016 10
Camera Model
z
yx
p
xr
xc
yx
f
• Camera extrinsics
• Camera intrinsics
• Projection
• Perspective compensation
• Lens distortion
LINK-SIC Presentation Clas Veibäck November 7, 2016 10
Camera Model
z
yx
p
xr
xc
yx
f
• Camera extrinsics
• Camera intrinsics
• Projection
• Perspective compensation
• Lens distortion
LINK-SIC Presentation Clas Veibäck November 7, 2016 10
Camera Model
z
yx
p
xr
xc
yx
f
• Camera extrinsics
• Camera intrinsics
• Projection
• Perspective compensation
• Lens distortion
LINK-SIC Presentation Clas Veibäck November 7, 2016 11
Dolphin Camera z
yx
p
xr
xc
yx
f
LINK-SIC Presentation Clas Veibäck November 7, 2016 12
Bird Camera z
yx
p
xr
xc
yx
f
1 Introduction2 Background3 Constrained Motion Model4 Uncertain Timestamp Model5 Mode Observations6 Conclusions
LINK-SIC Presentation Clas Veibäck November 7, 2016 14
Constrained Motion Model
LINK-SIC Presentation Clas Veibäck November 7, 2016 15
Constrained Motion Model
• Targets constrained to
region
• Feasible predictions
• Similar behaviour
LINK-SIC Presentation Clas Veibäck November 7, 2016 15
Constrained Motion Model
• Targets constrained to
region
• Feasible predictions
• Similar behaviour
LINK-SIC Presentation Clas Veibäck November 7, 2016 15
Constrained Motion Model
• Targets constrained to
region
• Feasible predictions
• Similar behaviour
LINK-SIC Presentation Clas Veibäck November 7, 2016 16
Turning Model
ω(x) = dr(x)
∫N
(βd + βa
(p⊥ · l(n)
))w(x, n) dn
• Nearly constant speed
• Influence by edges
• Avoid collision with edges
• Align with edges
• Preferred direction
• Polygon region
LINK-SIC Presentation Clas Veibäck November 7, 2016 16
Turning Model
ω(x) = dr(x)
∫N
(βd + βa
(p⊥ · l(n)
))w(x, n) dn
x =
(pp
)=
xyxy
• Nearly constant speed
• Influence by edges
• Avoid collision with edges
• Align with edges
• Preferred direction
• Polygon region
LINK-SIC Presentation Clas Veibäck November 7, 2016 16
Turning Model
ω(x) = dr(x)
∫N
(βd + βa
(p⊥ · l(n)
))w(x, n) dn
w(x, n) =1
‖p− n‖2
• Nearly constant speed
• Influence by edges
• Avoid collision with edges
• Align with edges
• Preferred direction
• Polygon region
LINK-SIC Presentation Clas Veibäck November 7, 2016 16
Turning Model
ω(x) = dr(x)
∫N
(βd + βa
(p⊥ · l(n)
))w(x, n) dn
• Nearly constant speed
• Influence by edges
• Avoid collision with edges
• Align with edges
• Preferred direction
• Polygon region
LINK-SIC Presentation Clas Veibäck November 7, 2016 16
Turning Model
ω(x) = dr(x)
∫N
(βd + βa
(p⊥ · l(n)
))w(x, n) dn
• Nearly constant speed
• Influence by edges
• Avoid collision with edges
• Align with edges
• Preferred direction
• Polygon region
LINK-SIC Presentation Clas Veibäck November 7, 2016 16
Turning Model
ω(x) = dr(x)
∫N
(βd + βa
(p⊥ · l(n)
))w(x, n) dn
Clockwise
or
Counterclockwise
• Nearly constant speed
• Influence by edges
• Avoid collision with edges
• Align with edges
• Preferred direction
• Polygon region
LINK-SIC Presentation Clas Veibäck November 7, 2016 16
Turning Model
ω(x) = dr(x)
N∑i=1
(βd + βa(p⊥ · li)
)wi(x)
l1lN
v1
v2
vN
©2015
IEEE
• Nearly constant speed
• Influence by edges
• Avoid collision with edges
• Align with edges
• Preferred direction
• Polygon region
LINK-SIC Presentation Clas Veibäck November 7, 2016 17
Potential Field
LINK-SIC Presentation Clas Veibäck November 7, 2016 18
Dolphin - Model Simulation
LINK-SIC Presentation Clas Veibäck November 7, 2016 19
Dolphin - Model Comparisons
Detection regionConstraint regionConstant Velocity modelCoordinated Turn modelConstrained Motion model
LINK-SIC Presentation Clas Veibäck November 7, 2016 20
Dolphin - Complete Framework
1 Introduction2 Background3 Constrained Motion Model4 Uncertain Timestamp Model5 Mode Observations6 Conclusions
LINK-SIC Presentation Clas Veibäck November 7, 2016 22
Uncertain Timestamp Model
LINK-SIC Presentation Clas Veibäck November 7, 2016 23
Uncertain Timestamp Model
• Traditional measurements
• Observations sampled at an
uncertain time
• Crime scene investigations
• Traces from animals
• Bottlenecks in mines
LINK-SIC Presentation Clas Veibäck November 7, 2016 23
Uncertain Timestamp Model
• Traditional measurements
• Observations sampled at an
uncertain time
• Crime scene investigations
• Traces from animals
• Bottlenecks in mines
LINK-SIC Presentation Clas Veibäck November 7, 2016 23
Uncertain Timestamp Model
• Traditional measurements
• Observations sampled at an
uncertain time
• Crime scene investigations
• Traces from animals
• Bottlenecks in mines
LINK-SIC Presentation Clas Veibäck November 7, 2016 23
Uncertain Timestamp Model
• Traditional measurements
• Observations sampled at an
uncertain time
• Crime scene investigations
• Traces from animals
• Bottlenecks in mines
LINK-SIC Presentation Clas Veibäck November 7, 2016 23
Uncertain Timestamp Model
• Traditional measurements
• Observations sampled at an
uncertain time
• Crime scene investigations
• Traces from animals
• Bottlenecks in mines
LINK-SIC Presentation Clas Veibäck November 7, 2016 24
Uncertain Timestamp Model
Consider a linear Gaussian state-space model,
xk = Fkxk−1 +wk, wk ∼ N (0,Qk),
yj = Hyjxj + vy
j , vyj ∼ N (0,Ry
j ),
x0 ∼ N (x0,P0),
Extend the model with
z = Hzxτ + vz, vz ∼ N (0,Rz), τ ∼ p(τ).
LINK-SIC Presentation Clas Veibäck November 7, 2016 24
Uncertain Timestamp Model
Consider a linear Gaussian state-space model,
xk = Fkxk−1 +wk, wk ∼ N (0,Qk),
yj = Hyjxj + vy
j , vyj ∼ N (0,Ry
j ),
x0 ∼ N (x0,P0),
Extend the model with
z = Hzxτ + vz, vz ∼ N (0,Rz), τ ∼ p(τ).
LINK-SIC Presentation Clas Veibäck November 7, 2016 25
Simple Uncertain Time Scenario
Consider the simple model,
xk = xk−1 + wk, wk ∼ N (0, Q),
yj = xj + vyj , vyj ∼ N (0, Ry)
for k ∈ {1, . . . , N} and two measurements y1 and yN .
Extend the model with
z = xτ + vz, vz ∼ N (0, Rz), τ ∼ p(τ).
LINK-SIC Presentation Clas Veibäck November 7, 2016 25
Simple Uncertain Time Scenario
Consider the simple model,
xk = xk−1 + wk, wk ∼ N (0, Q),
yj = xj + vyj , vyj ∼ N (0, Ry)
for k ∈ {1, . . . , N} and two measurements y1 and yN .
Extend the model with
z = xτ + vz, vz ∼ N (0, Rz), τ ∼ p(τ).
LINK-SIC Presentation Clas Veibäck November 7, 2016 26
Simple Uncertain Time Scenario
Time
0 0.2 0.4 0.6 0.8 1
Positio
n
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6Y
z (first case)z (second case)
The measurements are
• y1 = 0,
• yN = 1.
The two cases of
observations are
• z = 0.5 with flat prior.
• z = 1.5 with non-flatprior.
LINK-SIC Presentation Clas Veibäck November 7, 2016 26
Simple Uncertain Time Scenario
Time
0 0.2 0.4 0.6 0.8 1
Positio
n
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6Y
z (first case)z (second case)
The measurements are
• y1 = 0,
• yN = 1.
The two cases of
observations are
• z = 0.5 with flat prior.
• z = 1.5 with non-flatprior.
LINK-SIC Presentation Clas Veibäck November 7, 2016 27
Posterior Distributions - Time
The posterior distribution of the uncertain time is
wτ , p(τ |Y, z) ∝ p(τ)N (z| zτ , Sτ ).
LINK-SIC Presentation Clas Veibäck November 7, 2016 28
Posterior Distributions - Time
p(τ |Y, z)maxX p(X , τ |Y, z)p(τ)
τ
0 0.5 1
Pro
babili
ty
×10-3
0
0.5
1
1.5
2
τ
0 0.2 0.4 0.6 0.8 1
Pro
ba
bili
ty
×10-3
0
2
4
6
8
LINK-SIC Presentation Clas Veibäck November 7, 2016 29
Posterior Distributions - States
The posterior distribution of the state is
p(xk|Y, z) =
N∑τ=1
wτ · N (xk|xτk,P
τk).
LINK-SIC Presentation Clas Veibäck November 7, 2016 30
Posterior Distributions - States
Time
0 0.2 0.4 0.6 0.8 1
Po
sitio
n
-0.5
0
0.5
1
1.5
2
Time
0 0.2 0.4 0.6 0.8 1
Po
sitio
n
-0.5
0
0.5
1
1.5
2
LINK-SIC Presentation Clas Veibäck November 7, 2016 31
Estimators - Minimum Mean Squared Error
Yz
XMMSE |Y, z
XMMSE |Y
Time
0 0.5 1
Positio
n
0
0.5
1
1.5
Time
0 0.5 1
Positio
n
0
0.5
1
1.5
LINK-SIC Presentation Clas Veibäck November 7, 2016 32
Orienteering
• Adjusted trajectory
• Single control point
LINK-SIC Presentation Clas Veibäck November 7, 2016 32
Orienteering
• Adjusted trajectory
• Single control point
1 Introduction2 Background3 Constrained Motion Model4 Uncertain Timestamp Model5 Mode Observations6 Conclusions
LINK-SIC Presentation Clas Veibäck November 7, 2016 34
Mode Observations
M1
M2
M1
M2
M1
M2
k− 1 k k+ 1
LINK-SIC Presentation Clas Veibäck November 7, 2016 35
Mode Observations
M1
M2
M1
M2
M1
M2
k− 1 k k+ 1
• Jump Markov model
• Model target behaviour
• Probability of switching
• Direct mode observation
LINK-SIC Presentation Clas Veibäck November 7, 2016 35
Mode Observations
M1
M2
M1
M2
M1
M2
k− 1 k k+ 1
• Jump Markov model
• Model target behaviour
• Probability of switching
• Direct mode observation
LINK-SIC Presentation Clas Veibäck November 7, 2016 35
Mode Observations
M1
M2
M1
M2
M1
M2
k− 1 k k+ 1
• Jump Markov model
• Model target behaviour
• Probability of switching
• Direct mode observation
LINK-SIC Presentation Clas Veibäck November 7, 2016 35
Mode Observations
M1
M2
M1
M2
M1
M2
k− 1 k k+ 1
• Jump Markov model
• Model target behaviour
• Probability of switching
• Direct mode observation
LINK-SIC Presentation Clas Veibäck November 7, 2016 36
Jump Markov ModelThe jump Markov linear model is
xk = Fk(δk)xk−1 +wk, wk ∼ N (0, Qk(δk)),
yj = Hj(δj)xj + vj , vj ∼ N (0, Rj(δj)).
The mode is modelled as
p(δk | δk−1) = Πδk,δk−1
k , δk ∈ S.
Augmented is the direct mode observation,
zk ∼ p(zk | δk).
M1
M2
LINK-SIC Presentation Clas Veibäck November 7, 2016 36
Jump Markov ModelThe jump Markov linear model is
xk = Fk(δk)xk−1 +wk, wk ∼ N (0, Qk(δk)),
yj = Hj(δj)xj + vj , vj ∼ N (0, Rj(δj)).
The mode is modelled as
p(δk | δk−1) = Πδk,δk−1
k , δk ∈ S.
Augmented is the direct mode observation,
zk ∼ p(zk | δk).
M1
M2
LINK-SIC Presentation Clas Veibäck November 7, 2016 36
Jump Markov ModelThe jump Markov linear model is
xk = Fk(δk)xk−1 +wk, wk ∼ N (0, Qk(δk)),
yj = Hj(δj)xj + vj , vj ∼ N (0, Rj(δj)).
The mode is modelled as
p(δk | δk−1) = Πδk,δk−1
k , δk ∈ S.
Augmented is the direct mode observation,
zk ∼ p(zk | δk).
M1
M2
LINK-SIC Presentation Clas Veibäck November 7, 2016 36
Jump Markov ModelThe jump Markov linear model is
xk = Fk(δk)xk−1 +wk, wk ∼ N (0, Qk(δk)),
yj = Hj(δj)xj + vj , vj ∼ N (0, Rj(δj)).
The mode is modelled as
p(δk | δk−1) = Πδk,δk−1
k , δk ∈ S.
Augmented is the direct mode observation,
zk ∼ p(zk | δk).
M1
M2
LINK-SIC Presentation Clas Veibäck November 7, 2016 37
Posterior Distribution - Mode Sequence
The posterior probability of a mode sequence {δij}kj=1 is
computed recursively by
wik ∝ wi
k−1 ·Πδik,δ
ik−1
k · N (yk | yik, S
ik) · p(zk | δik).
M1
M2
LINK-SIC Presentation Clas Veibäck November 7, 2016 37
Posterior Distribution - Mode Sequence
The posterior probability of a mode sequence {δij}kj=1 is
computed recursively by
wik ∝ wi
k−1 ·Πδik,δ
ik−1
k · N (yk | yik, S
ik) · p(zk | δik).
M1
M2
LINK-SIC Presentation Clas Veibäck November 7, 2016 38
Posterior Distribution - State
The posterior distribution of the state is given by
p(xk | {yj}kj=1, {zj}kj=1) =
|S|k∑i=1
wik · N (xk | xi
k, Pik).
M1
M2
LINK-SIC Presentation Clas Veibäck November 7, 2016 39
Bird - ModelThe modes are S = {s, f}.
The state variables are x =
xybs
bf
=
pbs
bf
.
The state-space model is
xk = xk−1 +wk, wk ∼ N (0, Q(δk)),
yk =
(h(pk)
bδkk
)+ vk, vk ∼ N (0, R),
M1
M2
LINK-SIC Presentation Clas Veibäck November 7, 2016 39
Bird - ModelThe modes are S = {s, f}.
The state variables are x =
xybs
bf
=
pbs
bf
.
The state-space model is
xk = xk−1 +wk, wk ∼ N (0, Q(δk)),
yk =
(h(pk)
bδkk
)+ vk, vk ∼ N (0, R),
M1
M2
LINK-SIC Presentation Clas Veibäck November 7, 2016 39
Bird - ModelThe modes are S = {s, f}.
The state variables are x =
xybs
bf
=
pbs
bf
.
The state-space model is
xk = xk−1 +wk, wk ∼ N (0, Q(δk)),
yk =
(h(pk)
bδkk
)+ vk, vk ∼ N (0, R),
M1
M2
LINK-SIC Presentation Clas Veibäck November 7, 2016 40
Bird - Model ExtensionThe direct mode observation is the radial position
zk =√x2k + y2k,
where p(zk|δk) is given by:
Radius(mm)
0 100 200 300
Pro
babili
ty D
ensity
0
0.2
0.4
0.6
0.8
1
M1
M2
LINK-SIC Presentation Clas Veibäck November 7, 2016 40
Bird - Model ExtensionThe direct mode observation is the radial position
zk =√x2k + y2k,
where p(zk|δk) is given by:
Radius(mm)
0 100 200 300
Pro
babili
ty D
ensity
0
0.2
0.4
0.6
0.8
1
M1
M2
LINK-SIC Presentation Clas Veibäck November 7, 2016 41
Bird - Results
Time(s)
0 1000 2000 3000
Mode P
robabili
ty
0
0.5
1
Cage 1
Time(s)
0 1000 2000 3000
Mode P
robabili
ty
0
0.5
1
Cage 4
Time(s)
0 1000 2000 3000
Mode P
robabili
ty
0
0.5
1
Cage 2
Time(s)
0 1000 2000 3000
Mode P
robabili
ty
0
0.5
1
Cage 3
• Estimated modes
• Extracted takeoffs
• Takeoff directions
• Matched takeoffs
M1
M2
LINK-SIC Presentation Clas Veibäck November 7, 2016 41
Bird - Results
Time(s)
0 1000 2000 3000
Cage 1
Time(s)
0 1000 2000 3000
Cage 4
Time(s)
0 1000 2000 3000
Cage 2
Time(s)
0 1000 2000 3000
Cage 3
• Estimated modes
• Extracted takeoffs
• Takeoff directions
• Matched takeoffs
M1
M2
LINK-SIC Presentation Clas Veibäck November 7, 2016 41
Bird - Results
Cage 1 Cage 4
Cage 2 Cage 3
• Estimated modes
• Extracted takeoffs
• Takeoff directions
• Matched takeoffs
M1
M2
LINK-SIC Presentation Clas Veibäck November 7, 2016 41
Bird - Results
Cage 1 Cage 4
Cage 2 Cage 3
• Estimated modes
• Extracted takeoffs
• Takeoff directions
• Matched takeoffs
M1
M2
LINK-SIC Presentation Clas Veibäck November 7, 2016 42
Bird - Complete Framework M1
M2
1 Introduction2 Background3 Constrained Motion Model4 Uncertain Timestamp Model5 Mode Observations6 Conclusions
LINK-SIC Presentation Clas Veibäck November 7, 2016 44
Conclusions
Theory is presented on
• a constrained motion model
• an uncertain timestamp model
• mode observations
Theory is demonstrated to work in applications
LINK-SIC Presentation Clas Veibäck November 7, 2016 44
Conclusions
Theory is presented on
• a constrained motion model
• an uncertain timestamp model
• mode observations
Theory is demonstrated to work in applications
LINK-SIC Presentation Clas Veibäck November 7, 2016 44
Conclusions
Theory is presented on
• a constrained motion model
• an uncertain timestamp model
• mode observations
Theory is demonstrated to work in applications
LINK-SIC Presentation Clas Veibäck November 7, 2016 44
Conclusions
Theory is presented on
• a constrained motion model
• an uncertain timestamp model
• mode observations
Theory is demonstrated to work in applications
Questions?www.liu.se