Shadow Resistant Video Tracking

Hao Jiang and Mark S. Drew

School of Computing Science

Simon Fraser University

Vancouver, BC, Canada

Problem Statement

Traditional ContourTracking based onmotions

We want to realize this!

Outline

We present an invariant image model. We study how to project an image to an invariant space, such that the shadow can be greatly attenuated.

We present two new external forces to the snake model and present an chordal snake model to deal with object tracking in cluttering environment.

The first external force is based on predictive contour

The chordal constraint based on a new shape descriptor

Results and conclusion

Invariant Image

T

c

ii ecIE 2

51)(

kT

c

kkii

kik

qeSIc

dQSEx

k

2

51

0

)()(

)()()()(

xna

For narrow bandSensors:

n

ai

Lambertian Surface

)()( kkk qQ

The responses:

Planckian Lighting

x

Considering 3-sensor cameras, = R, = G, = B

Let r=log(R/G) , b=log(B/G)

1 2 3

2313

12135

122

5211

5322

5233 )]

)(

)(log([)

)(

)(log(

Sq

Sqr

Sq

Sqb

r

b

The slope is determined by the camera sensors

ref-1

ref-2

…

Lighting

Material

We get,

Invariant Image Generation

r

b

Invariant Image Generation

o

Camera characteristicorientation

(r,g,b)

(log(r/g), log(b/g))

Projection

Camera Calibration

Take image of one scene under different lightings

Shift the center ofthe log-log ratios corresponding

to each material to the origin

Stack the log-ratio vectors of each material

into a matrix A and do SVD A=UDV’Camera Orientation=

V(:,1) Characteristic Orientation of Canon ES60

For Real Image

Original image Invariant image

Inertia Snake Tracking I

A predictive contour constraint

s

dssXPsCsXEsXsX ))(())(),((2

|)(|2

|)(|2

min 222

0)()( XCXPXX ssssss

If we choose quadratic norm for E(.,.) the Eular Equation,

)()( XCXFXXt

Xssssss

By introducing a artificial parameter t, the equation can be solved by PDE

Inertia Term

A Chordal ConstraintNow we further introduce a second constraint to maintain the solidness of the shape of the contour by maintaining the value of a shape descriptor.The shape discriptor is defined as

d(s, )=||X(s)-X(s+ )|| where s in is the normalized length from one point on the contour.

Apparently, d(x, ) is periodic for both s and .

s

0 1

1

The Shape descriptor

2/11

0

1

0

2

1

0

1

0)1,0[

]),([)(

])(

),(

)(

),([max),(

dsdsddStd

dsddStd

sd

dStd

sdYXSimilarity

Y

Y

X

X

ddDStd

D

DStd

DYXSimilarity s

Y

sY

X

sX ]||)(||

||),(||

||)(||

||),(||[),(

1

0

1

0

The similarity of contour X and Y is,

In frequency domain

0 1

1s

The Chordal Snake

dssdsYsXG

sYPsXPsDsYEsCsXE

sYsXsYsXs

))(||,)()((||

))(()(())(),(())(),(((2

)|)(||)((|2

)|)(||)((|2

min 222222

Here we use a simple version d(s)=||X(s)-X(s+1/2)||

The variational problem is

Where Y(s) is an accessory contour, d(s) is the calculated fromlast video frame. The corresponding reaction-diffusion PDE is:

)||(||||||

)()()(

)||(||||||

)()()(

dYXYX

YXYDYFYY

t

Y

dYXYX

XYXCXFXX

t

X

ssssss

ssssss

The Chordal Snake

Now we set Y0(s)= X0(s+1/2), D0(s)= C0(s+1/2). It is not difficultto prove the following Lemma and theorem.

Lemma: If Y(s,t1)= X(s+1/2,t1), D(s,t1)= C(s+1/2,t1) then Y(s,t)= X(s+1/2,t) for any t>= t1

Theorem: Given the initial conditions of Y0(s)= X0(s+1/2), D0(s)= C0(s+1/2), we have,

)||),2/1((||||),2/1(||

)),(),2/1((

)()(

dtsXXtsXX

tsXtsX

XCXFXXt

Xssssss

Predictive Constraint

Shape descriptor constraint

Chordal Snake Tracking II

Smoothedpredictive contour

Initial contourPrevious contour

Predictive contour

Real object boundary

Features

Chordal constraint

The System

Affine Motion Estimation

warping

Fn

Fn-1

Motion Detection

InvariantImage

InvariantImage

Motion Detection

Inertia Snake Tracking

MotionMap

ContourPrediction

Cn-1 GVF

Init Contour Pred Contour External Force

CnChordal Model

An Example

Initial Contour

Prediction Contour

Experiment Result

Two successive frames

Motion map in original color space Motion map in invariant color space

Experiment Result

Two successive frames

Motion map in original color space Motion map in invariant color space

Traditional Snake Model

Frame 1 Frame 2 Frame 3

Frame 4 Frame 5 Frame 6

Frame 7 Frame 8 Frame 9

Tracking Result

Ball Sequence Hand Sequence Baby Sequence

Conclusion

We present scheme to get shadow invariant image.

We present a much more robust snake model.

The proposed method can work well even though there is strong distracting shadows

Current framework can be easily extended to the cases when the object is passing casting shadows

Future WorkStudy scheme to deal with tracking in high dynamic range environment

Study shadow resistant method for active appear model