ee392j final project, march 20, 2002 1 multiple camera object tracking helmy eltoukhy and khaled...

EE392J Final Project, March 20, 2002 1

Multiple Camera Object TrackingMultiple Camera Object Tracking

Helmy Eltoukhy and Khaled SalamaHelmy Eltoukhy and Khaled Salama


OutlineOutline

IntroductionIntroduction Point Correspondence between multiple Point Correspondence between multiple

camerascameras Robust Object TrackingRobust Object Tracking Camera Communication and decision Camera Communication and decision

makingmaking ResultsResults


Object TrackingObject Tracking

The objective is to obtain an accurate The objective is to obtain an accurate estimate of the position (x,y) of the object estimate of the position (x,y) of the object trackedtracked

Tracking algorithms can be classified intoTracking algorithms can be classified into• Single object & Single Camera Single object & Single Camera • Multiple object & Single Camera Multiple object & Single Camera • Multiple objects & Multiple CamerasMultiple objects & Multiple Cameras• Single object & Multiple CamerasSingle object & Multiple Cameras


Single Object & Single CameraSingle Object & Single Camera

Accurate camera calibration and scene modelAccurate camera calibration and scene model Suffers from OcclusionsSuffers from Occlusions Not robust and object dependantNot robust and object dependant


Single Object & Multiple CameraSingle Object & Multiple Camera Accurate point correspondence between scenesAccurate point correspondence between scenes Occlusions can be minimized or even avoidedOcclusions can be minimized or even avoided Redundant information for better estimationRedundant information for better estimation Multiple camera Communication problemMultiple camera Communication problem


System ArchitectureSystem Architecture

ObjectIdentification

ObjectTracking

Check Position(X1-X2) <(Y1-Y2) <

ChooseCameraView

ObjectIdentification

ObjectTracking


Static Point CorrespondenceStatic Point Correspondence

The output of the tracking stage isThe output of the tracking stage is A simple scene model is used to get real A simple scene model is used to get real

estimate of coordinates estimate of coordinates Both Affine and Perspective models were Both Affine and Perspective models were

used for the scene modeling and static used for the scene modeling and static corresponding points were used for corresponding points were used for parameter estimationparameter estimation

Least mean squares was used to improve Least mean squares was used to improve parameter estimationparameter estimation

)(),( nYnX ii

)(ˆ),(ˆ nYnX ii


Dynamic Point CorrespondenceDynamic Point Correspondence

Affine modelusing A(n)

B(n)

Affine modelusing A(n)

Add this point to AAdd this point to A

Check Position(X1-X2) < T(Y1-Y2) < T


Block-Based Motion EstimationBlock-Based Motion Estimation

Typically, in object tracking precise sub-pixel Typically, in object tracking precise sub-pixel optical flow estimation is not needed.optical flow estimation is not needed.

Furthermore, motion can be on the order of Furthermore, motion can be on the order of several pixels, thereby precluding use of gradient several pixels, thereby precluding use of gradient methods.methods.

We started with a simple sum of squared We started with a simple sum of squared differences error criterion coupled with full search differences error criterion coupled with full search in a limited region around the tracking window.in a limited region around the tracking window.

• 2

),(

)),,(),,(( tyxsttnymxsSSD cyx

cerror


Adaptive Window SizingAdaptive Window Sizing

Although simple block-based motion Although simple block-based motion estimation may work reasonably well when estimation may work reasonably well when motion is purely translational, it can lose the motion is purely translational, it can lose the object if its relative size changes.object if its relative size changes.• If the object’s camera field of view shrinks, the If the object’s camera field of view shrinks, the

SSD error is strongly influenced by the SSD error is strongly influenced by the background.background.

• If the object’s camera field of view grows, the If the object’s camera field of view grows, the window fails to make use of entire object window fails to make use of entire object information and can slip away. information and can slip away.


Four Corner MethodFour Corner Method

This technique divides the rectangular object This technique divides the rectangular object window into 4 basic regions - each one quadrant.window into 4 basic regions - each one quadrant.

Motion vectors are calculated for each subregion Motion vectors are calculated for each subregion and each controls one of four corners.and each controls one of four corners.

Translational motion is captured by all four moving Translational motion is captured by all four moving equally, while window size is modulated when equally, while window size is modulated when motion is differential.motion is differential.

Resultant tracking window can be non-rectangular, Resultant tracking window can be non-rectangular, i.e., any quadrilateral approximated by four i.e., any quadrilateral approximated by four rectangles with a shared center corner.rectangles with a shared center corner.


Example: Four Corner MethodExample: Four Corner Method

Synthetically generated test sequences:


Correlative MethodCorrelative Method

Four corner method is strongly subject to error Four corner method is strongly subject to error accumulation which can result in drift of one or accumulation which can result in drift of one or more of the tracking window quadrants.more of the tracking window quadrants.

Once drift occurs, sizing of window is highly Once drift occurs, sizing of window is highly inaccurate.inaccurate.

Need a method that has some corrective feedback Need a method that has some corrective feedback so window can converge to correct size even after so window can converge to correct size even after some errors.some errors.

Correlation of current object features to some Correlation of current object features to some template view is one solution.template view is one solution.


Correlative Method (con’t)Correlative Method (con’t)

Basic form of technique involves storing initial Basic form of technique involves storing initial view of object as a reference image.view of object as a reference image.

Block matching is performed through a combined Block matching is performed through a combined interframe and correlative MSE:interframe and correlative MSE:

•

where sc’(x0,y0,0) is the resized stored template image.where sc’(x0,y0,0) is the resized stored template image.

Furthermore, minimum correlative MSE is used to Furthermore, minimum correlative MSE is used to direct resizing of current window.direct resizing of current window.

,))0,,(),,(()1(

)),,(),,((1

200

'

),(1

2

),(1

21

yxsttnymxsa

tyxsttnymxsann

MSE

c

yxc

cyx

cerror


Example: Correlative MethodExample: Correlative Method


Occlusion DetectionOcclusion Detection

In order for multi-camera feature tracking to work, In order for multi-camera feature tracking to work, each camera must possess an ability to assess the each camera must possess an ability to assess the validity of its tracking (e.g. to detect occlusion).validity of its tracking (e.g. to detect occlusion).

Comparing the minimum error at each point to Comparing the minimum error at each point to some absolute threshold is problematic since error some absolute threshold is problematic since error can grow even when tracking is still valid.can grow even when tracking is still valid.

Threshold must be adaptive to current conditions.Threshold must be adaptive to current conditions. One solution is to use a threshold of k (constant > One solution is to use a threshold of k (constant >

1) times the moving average of the MSE.1) times the moving average of the MSE. Thus, only precipitous changes in error trigger Thus, only precipitous changes in error trigger

indication of possibly fallacious tracking. indication of possibly fallacious tracking.


Redetection Procedure (1 Camera)Redetection Procedure (1 Camera)

Normal Tracking Occlusion Detected

Motion Tracking Remain Stationary

It < dNoiseIt > dNoiseError < Erravg

Error > kErravg

Redetection is difficult at most general level Redetection is difficult at most general level – Object recognition.– Object recognition.

Proximity and size constancy constraints can Proximity and size constancy constraints can be imposed to simplify redetection. be imposed to simplify redetection.


Example: OcclusionExample: Occlusion


Camera CommunicationCamera Communication


ResultResult


ConclusionConclusion

Multiple cameras can do more than just 3D Multiple cameras can do more than just 3D imagingimaging

Camera calibration only works if you have Camera calibration only works if you have an accurate scene and camera modelan accurate scene and camera model

Tracking is sensitive to the camera Tracking is sensitive to the camera characteristics (noise, blur, frame rate,..)characteristics (noise, blur, frame rate,..)

Tracking accuracy can be improved using Tracking accuracy can be improved using multiple camerasmultiple cameras

ee392j final project, march 20, 2002 1 multiple camera object tracking helmy eltoukhy and khaled...

Documents