two examples of indoor and outdoor surveillance systems: motivation, design, and testing
DESCRIPTION
Two Examples Of Indoor And Outdoor Surveillance Systems: Motivation, Design, And Testing. Ioannis Pavlidis Vassilios Morellas Honeywell Laboratories. Graduate Seminar in CIS Video Processing and Mining CIS 750 – Spring 2003. Presented by: Ken Gorman. Agenda. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Two Examples Of Indoor And Outdoor Surveillance Systems: Motivation, Design, And Testing](https://reader036.vdocuments.site/reader036/viewer/2022062518/5681402a550346895dab8b29/html5/thumbnails/1.jpg)
Two Examples Of Indoor And OutdoorTwo Examples Of Indoor And OutdoorSurveillance Systems:Surveillance Systems:
Motivation, Design, And TestingMotivation, Design, And Testing
Ioannis PavlidisIoannis Pavlidis
Vassilios MorellasVassilios Morellas
Honeywell LaboratoriesHoneywell Laboratories
![Page 2: Two Examples Of Indoor And Outdoor Surveillance Systems: Motivation, Design, And Testing](https://reader036.vdocuments.site/reader036/viewer/2022062518/5681402a550346895dab8b29/html5/thumbnails/2.jpg)
Graduate Seminar in CISGraduate Seminar in CISVideo Processing and MiningVideo Processing and Mining
CIS 750 – Spring 2003CIS 750 – Spring 2003
Presented by:Presented by:
Ken GormanKen Gorman
![Page 3: Two Examples Of Indoor And Outdoor Surveillance Systems: Motivation, Design, And Testing](https://reader036.vdocuments.site/reader036/viewer/2022062518/5681402a550346895dab8b29/html5/thumbnails/3.jpg)
AgendaAgenda
CCN – Cooperative Camera NetworkCCN – Cooperative Camera Network
DETER - Detection of Events for Threat DETER - Detection of Events for Threat Evaluation and RecognitionEvaluation and Recognition
![Page 4: Two Examples Of Indoor And Outdoor Surveillance Systems: Motivation, Design, And Testing](https://reader036.vdocuments.site/reader036/viewer/2022062518/5681402a550346895dab8b29/html5/thumbnails/4.jpg)
Cooperative Camera Network (CCN)Cooperative Camera Network (CCN)
Network of cooperating cameras Network of cooperating cameras Controlled by computer vision softwareControlled by computer vision software Features:Features:
– mechanism for counting the people present in various mechanism for counting the people present in various parts of the buildingparts of the building
– An automatic or semi-automatic mechanism for tagging An automatic or semi-automatic mechanism for tagging people. people.
– Report tagged individuals whereabouts whenever they Report tagged individuals whereabouts whenever they are within the field of vieware within the field of view
![Page 5: Two Examples Of Indoor And Outdoor Surveillance Systems: Motivation, Design, And Testing](https://reader036.vdocuments.site/reader036/viewer/2022062518/5681402a550346895dab8b29/html5/thumbnails/5.jpg)
Major ComponentsMajor Components
COTS Hardware & Software Set-UpCOTS Hardware & Software Set-Up Change DetectionChange Detection Counting People Counting People Tracking PeopleTracking People
![Page 6: Two Examples Of Indoor And Outdoor Surveillance Systems: Motivation, Design, And Testing](https://reader036.vdocuments.site/reader036/viewer/2022062518/5681402a550346895dab8b29/html5/thumbnails/6.jpg)
State of the ArtState of the Art
Active badges Active badges – small, electronic devices worn by peoplesmall, electronic devices worn by people– transmit an ID signal to receivers placed around transmit an ID signal to receivers placed around
the buildingthe building– ID signal corresponds to the identity of the ID signal corresponds to the identity of the
badge’s wearerbadge’s wearer– received signals are used to compute the received signals are used to compute the
wearer’s locationwearer’s location
![Page 7: Two Examples Of Indoor And Outdoor Surveillance Systems: Motivation, Design, And Testing](https://reader036.vdocuments.site/reader036/viewer/2022062518/5681402a550346895dab8b29/html5/thumbnails/7.jpg)
Badge ExamplesBadge Examples
Infrared-transmitting badges at Olivetti Infrared-transmitting badges at Olivetti Research and Xerox PARC, Research and Xerox PARC,
Olivetti ultrasonic badges at AT&T Olivetti ultrasonic badges at AT&T Laboratories in Cambridge, UK Laboratories in Cambridge, UK
Radio frequency tags from PinPointRadio frequency tags from PinPoint Wired and unwired motion trackers fromWired and unwired motion trackers from
– Ascension Technology Ascension Technology – PolhemusPolhemus
![Page 8: Two Examples Of Indoor And Outdoor Surveillance Systems: Motivation, Design, And Testing](https://reader036.vdocuments.site/reader036/viewer/2022062518/5681402a550346895dab8b29/html5/thumbnails/8.jpg)
DisadvantagesDisadvantages
Consumers unwilling to wear badgesConsumers unwilling to wear badges CumbersomeCumbersome
![Page 9: Two Examples Of Indoor And Outdoor Surveillance Systems: Motivation, Design, And Testing](https://reader036.vdocuments.site/reader036/viewer/2022062518/5681402a550346895dab8b29/html5/thumbnails/9.jpg)
AlternativesAlternatives
????????
![Page 10: Two Examples Of Indoor And Outdoor Surveillance Systems: Motivation, Design, And Testing](https://reader036.vdocuments.site/reader036/viewer/2022062518/5681402a550346895dab8b29/html5/thumbnails/10.jpg)
CamerasCameras
Pro – Leaves users unencumberedPro – Leaves users unencumbered Cons – Not as reliable as badge methodsCons – Not as reliable as badge methods
![Page 11: Two Examples Of Indoor And Outdoor Surveillance Systems: Motivation, Design, And Testing](https://reader036.vdocuments.site/reader036/viewer/2022062518/5681402a550346895dab8b29/html5/thumbnails/11.jpg)
Camera ArrangementCamera Arrangement
Overlapping Fields of ViewOverlapping Fields of View
![Page 12: Two Examples Of Indoor And Outdoor Surveillance Systems: Motivation, Design, And Testing](https://reader036.vdocuments.site/reader036/viewer/2022062518/5681402a550346895dab8b29/html5/thumbnails/12.jpg)
FundamentalsFundamentals
Imaging Technologies for Surveillance Imaging Technologies for Surveillance SystemsSystems– Image SegmentationImage Segmentation– Tracking MechanismTracking Mechanism– Multi-Camera FusionMulti-Camera Fusion
Threat AssessmentThreat Assessment
![Page 13: Two Examples Of Indoor And Outdoor Surveillance Systems: Motivation, Design, And Testing](https://reader036.vdocuments.site/reader036/viewer/2022062518/5681402a550346895dab8b29/html5/thumbnails/13.jpg)
Multi-Normal Pixel RepresentationMulti-Normal Pixel Representation
![Page 14: Two Examples Of Indoor And Outdoor Surveillance Systems: Motivation, Design, And Testing](https://reader036.vdocuments.site/reader036/viewer/2022062518/5681402a550346895dab8b29/html5/thumbnails/14.jpg)
InitializationInitialization
Goal - provide statistically valid values for Goal - provide statistically valid values for the pixels corresponding to the scene.the pixels corresponding to the scene.
Starting point for the dynamic process of Starting point for the dynamic process of foreground and background awarenessforeground and background awareness
![Page 15: Two Examples Of Indoor And Outdoor Surveillance Systems: Motivation, Design, And Testing](https://reader036.vdocuments.site/reader036/viewer/2022062518/5681402a550346895dab8b29/html5/thumbnails/15.jpg)
InitializationInitialization
Methods Used:Methods Used:– K-Means – better for plazas and mallsK-Means – better for plazas and malls– Expectation-Maximization – better for Expectation-Maximization – better for
changing weather conditions [1]changing weather conditions [1]
![Page 16: Two Examples Of Indoor And Outdoor Surveillance Systems: Motivation, Design, And Testing](https://reader036.vdocuments.site/reader036/viewer/2022062518/5681402a550346895dab8b29/html5/thumbnails/16.jpg)
Image SegmentationImage Segmentation
Each pixel is considered as a mixture of Each pixel is considered as a mixture of five time-varying trivariate normal five time-varying trivariate normal distributionsdistributions
)(
15,,1,0
,~
5
1
5
13
weightssproportionmixingare
andi
where
Nx
iii
iiii
![Page 17: Two Examples Of Indoor And Outdoor Surveillance Systems: Motivation, Design, And Testing](https://reader036.vdocuments.site/reader036/viewer/2022062518/5681402a550346895dab8b29/html5/thumbnails/17.jpg)
Image SegmentationImage Segmentation
The term represents a trivariate Normal The term represents a trivariate Normal distribution with vector mean and variance-distribution with vector mean and variance-covariance matrixcovariance matrix
iiN ,3
![Page 18: Two Examples Of Indoor And Outdoor Surveillance Systems: Motivation, Design, And Testing](https://reader036.vdocuments.site/reader036/viewer/2022062518/5681402a550346895dab8b29/html5/thumbnails/18.jpg)
Image SegmentationImage Segmentation
The distributions are trivariate to account The distributions are trivariate to account for the three component colors (red, green, for the three component colors (red, green, and blue) of each pixel in the general case and blue) of each pixel in the general case of a color camera.of a color camera.
B
G
R
x
x
x
x
![Page 19: Two Examples Of Indoor And Outdoor Surveillance Systems: Motivation, Design, And Testing](https://reader036.vdocuments.site/reader036/viewer/2022062518/5681402a550346895dab8b29/html5/thumbnails/19.jpg)
Image SegmentationImage Segmentation
For simplification, the variance-covariance For simplification, the variance-covariance matrix is assumed to be diagonal with matrix is assumed to be diagonal with xxRR,x,xGG,x,xBB , having identical variance within , having identical variance within each Normal component, but not across all each Normal component, but not across all componentscomponents
INx iBi
Gi
Ri
ii
25
13 ,~
![Page 20: Two Examples Of Indoor And Outdoor Surveillance Systems: Motivation, Design, And Testing](https://reader036.vdocuments.site/reader036/viewer/2022062518/5681402a550346895dab8b29/html5/thumbnails/20.jpg)
Update CycleUpdate Cycle
Distributions are ordered based upon their Distributions are ordered based upon their weights.weights.
Member of a DistributionMember of a Distribution Distribution is in background or foregroundDistribution is in background or foreground Jeffreys [2] algorithm used for “matching” Jeffreys [2] algorithm used for “matching”
pixel to distributionpixel to distribution Distributions are UpdatedDistributions are Updated
![Page 21: Two Examples Of Indoor And Outdoor Surveillance Systems: Motivation, Design, And Testing](https://reader036.vdocuments.site/reader036/viewer/2022062518/5681402a550346895dab8b29/html5/thumbnails/21.jpg)
MatchingMatching
We use the Jeffreys divergence measure (J) to determine whether the We use the Jeffreys divergence measure (J) to determine whether the incoming data point belongs to one of the existing distributionsincoming data point belongs to one of the existing distributions
The Jeffreys number measures how unlikely it is that one distribution The Jeffreys number measures how unlikely it is that one distribution (g) was drawn from the population represented by the other (f)(g) was drawn from the population represented by the other (f)
K* - prespecified cut-off valueK* - prespecified cut-off value
*,
,,
11
2
1
2
3,
min51
22
2
KgfJ
gfJgfJ
uuuugfJ
i
ii
i
gggg
gii
g
g
ii
![Page 22: Two Examples Of Indoor And Outdoor Surveillance Systems: Motivation, Design, And Testing](https://reader036.vdocuments.site/reader036/viewer/2022062518/5681402a550346895dab8b29/html5/thumbnails/22.jpg)
Update – Match FoundUpdate – Match Found
Incoming pixel state is labeled either Incoming pixel state is labeled either background or foreground background or foreground
All the parameters of the matched All the parameters of the matched distribution are updated according to the distribution are updated according to the method of momentsmethod of moments
Only the weights of the other distributions Only the weights of the other distributions are updatedare updated
![Page 23: Two Examples Of Indoor And Outdoor Surveillance Systems: Motivation, Design, And Testing](https://reader036.vdocuments.site/reader036/viewer/2022062518/5681402a550346895dab8b29/html5/thumbnails/23.jpg)
Update – No MatchUpdate – No Match
Incoming pixel state is labeled foreground Incoming pixel state is labeled foreground Last distribution in the ordered list is Last distribution in the ordered list is
replaced replaced All the parameters of the new distribution All the parameters of the new distribution
are updated are updated Only the weights of the other distributions Only the weights of the other distributions
are updatedare updated
![Page 24: Two Examples Of Indoor And Outdoor Surveillance Systems: Motivation, Design, And Testing](https://reader036.vdocuments.site/reader036/viewer/2022062518/5681402a550346895dab8b29/html5/thumbnails/24.jpg)
Jeffrey’s AlgorithmJeffrey’s Algorithm
Jeffreys number measures how unlikely it is Jeffreys number measures how unlikely it is that one distribution (that one distribution (gg) was drawn from the ) was drawn from the population represented by the other (population represented by the other (ff).).
![Page 25: Two Examples Of Indoor And Outdoor Surveillance Systems: Motivation, Design, And Testing](https://reader036.vdocuments.site/reader036/viewer/2022062518/5681402a550346895dab8b29/html5/thumbnails/25.jpg)
Broken CloudsBroken Clouds
Preferential, in orderPreferential, in order No ProferenceNo Proference
![Page 26: Two Examples Of Indoor And Outdoor Surveillance Systems: Motivation, Design, And Testing](https://reader036.vdocuments.site/reader036/viewer/2022062518/5681402a550346895dab8b29/html5/thumbnails/26.jpg)
Segmentation of Moving ObjectsSegmentation of Moving Objects
The form of some of the distributions could The form of some of the distributions could changechange
Some of the foreground states could revert Some of the foreground states could revert to back-ground and vice versa.to back-ground and vice versa.
One of the existing distributions could be One of the existing distributions could be dropped and replaced with a new dropped and replaced with a new distribution.distribution.
![Page 27: Two Examples Of Indoor And Outdoor Surveillance Systems: Motivation, Design, And Testing](https://reader036.vdocuments.site/reader036/viewer/2022062518/5681402a550346895dab8b29/html5/thumbnails/27.jpg)
Predictive TrackingPredictive Tracking
On-line segmentation of foreground pixelsOn-line segmentation of foreground pixels Calculation of blob centroidsCalculation of blob centroids Multiple-Hypotheses Tracking AlgorithmMultiple-Hypotheses Tracking Algorithm
![Page 28: Two Examples Of Indoor And Outdoor Surveillance Systems: Motivation, Design, And Testing](https://reader036.vdocuments.site/reader036/viewer/2022062518/5681402a550346895dab8b29/html5/thumbnails/28.jpg)
Multiple-Hypotheses TrackingMultiple-Hypotheses Tracking
Recursive Bayesian probabilistic procedureRecursive Bayesian probabilistic procedure Does NOT commit early to trajectoryDoes NOT commit early to trajectory
![Page 29: Two Examples Of Indoor And Outdoor Surveillance Systems: Motivation, Design, And Testing](https://reader036.vdocuments.site/reader036/viewer/2022062518/5681402a550346895dab8b29/html5/thumbnails/29.jpg)
Multiple Hypothesis TrackingMultiple Hypothesis Tracking
![Page 30: Two Examples Of Indoor And Outdoor Surveillance Systems: Motivation, Design, And Testing](https://reader036.vdocuments.site/reader036/viewer/2022062518/5681402a550346895dab8b29/html5/thumbnails/30.jpg)
Multiple Hypothesis TrackingMultiple Hypothesis Tracking
Kalman filtering prediction based on constant Kalman filtering prediction based on constant velocity models velocity models
K-best hypothesis trajectory tree generation, K-best hypothesis trajectory tree generation, pruning and merging pruning and merging
Bayesian probability calculations for matching Bayesian probability calculations for matching input data to track hypothesisinput data to track hypothesis
See references [5] and [6] for exact algorithmSee references [5] and [6] for exact algorithm
![Page 31: Two Examples Of Indoor And Outdoor Surveillance Systems: Motivation, Design, And Testing](https://reader036.vdocuments.site/reader036/viewer/2022062518/5681402a550346895dab8b29/html5/thumbnails/31.jpg)
Multi-Camera FusionMulti-Camera Fusion
Monitoring of large areas can only be Monitoring of large areas can only be accomplished using multiple camerasaccomplished using multiple cameras
Panoramic View is created by fusing Panoramic View is created by fusing individual camera FOVsindividual camera FOVs
Object motion registered against a global Object motion registered against a global coordinate systemcoordinate system
![Page 32: Two Examples Of Indoor And Outdoor Surveillance Systems: Motivation, Design, And Testing](https://reader036.vdocuments.site/reader036/viewer/2022062518/5681402a550346895dab8b29/html5/thumbnails/32.jpg)
Multi-Camera FusionMulti-Camera Fusion
Optimal Coverage Scheme is createdOptimal Coverage Scheme is created Minimal use of cameras to minimize costMinimal use of cameras to minimize cost
![Page 33: Two Examples Of Indoor And Outdoor Surveillance Systems: Motivation, Design, And Testing](https://reader036.vdocuments.site/reader036/viewer/2022062518/5681402a550346895dab8b29/html5/thumbnails/33.jpg)
Multi-Camera FusionMulti-Camera Fusion
Compute HOMOGRAPHY matrix H Compute HOMOGRAPHY matrix H between two cameras based on CoG of between two cameras based on CoG of moving objects appearing in the moving objects appearing in the overlapping areas of the two fields of view overlapping areas of the two fields of view
Requirement: Information exchange Requirement: Information exchange between respective computers (e.g., pixel between respective computers (e.g., pixel intensity data and CoG of moving objects in intensity data and CoG of moving objects in pixel coordinates) pixel coordinates)
![Page 34: Two Examples Of Indoor And Outdoor Surveillance Systems: Motivation, Design, And Testing](https://reader036.vdocuments.site/reader036/viewer/2022062518/5681402a550346895dab8b29/html5/thumbnails/34.jpg)
Homography Matrices ComputationHomography Matrices Computation
Least Squares MethodLeast Squares Method Very popularVery popular Relatively simple Relatively simple Defined in Reference [6]Defined in Reference [6]
![Page 35: Two Examples Of Indoor And Outdoor Surveillance Systems: Motivation, Design, And Testing](https://reader036.vdocuments.site/reader036/viewer/2022062518/5681402a550346895dab8b29/html5/thumbnails/35.jpg)
Homography MatrixHomography Matrix
Used Kanatani MethodUsed Kanatani Method Based on a statistical optimization theory for Based on a statistical optimization theory for
geometric computer visiongeometric computer vision Cures the deficiencies exhibited by Least-SquaresCures the deficiencies exhibited by Least-Squares
![Page 36: Two Examples Of Indoor And Outdoor Surveillance Systems: Motivation, Design, And Testing](https://reader036.vdocuments.site/reader036/viewer/2022062518/5681402a550346895dab8b29/html5/thumbnails/36.jpg)
Kanatani MethodKanatani Method
Epipolar constraint may be violated by various Epipolar constraint may be violated by various noise sources due to the statistical nature of the noise sources due to the statistical nature of the imaging problemimaging problem
![Page 37: Two Examples Of Indoor And Outdoor Surveillance Systems: Motivation, Design, And Testing](https://reader036.vdocuments.site/reader036/viewer/2022062518/5681402a550346895dab8b29/html5/thumbnails/37.jpg)
Multi-Camera FusionMulti-Camera Fusion
OO1 1 andand OO22 are Optical Centers are Optical Centers
P(x,y,z) is a point in the scene that falls in the P(x,y,z) is a point in the scene that falls in the common area between the two cameracommon area between the two camera
Vector OVector O11p, Op, O22q, and Oq, and O11OO2 2 are co-planarare co-planar
![Page 38: Two Examples Of Indoor And Outdoor Surveillance Systems: Motivation, Design, And Testing](https://reader036.vdocuments.site/reader036/viewer/2022062518/5681402a550346895dab8b29/html5/thumbnails/38.jpg)
ReferencesReferences
Paolo Remagnino , et al (Editors). Paolo Remagnino , et al (Editors). Video-Based Surveillance Systems: Video-Based Surveillance Systems: Computer Vision and Distributed ProcessingComputer Vision and Distributed Processing. Kluwer Academic Publishers, . Kluwer Academic Publishers, 2002.2002.
http://www.htc.honeywell.com/projects/deter/http://www.htc.honeywell.com/projects/deter/ [1] - A. P. Dempster, N. M. Laird, and D. B. Rubin, “Maximum likelihood [1] - A. P. Dempster, N. M. Laird, and D. B. Rubin, “Maximum likelihood
from incomplete data via the em algorithm (with discussion),” J. Roy. Stat. from incomplete data via the em algorithm (with discussion),” J. Roy. Stat. Soc. B, vol. 39, pp. 1–38, 1977.Soc. B, vol. 39, pp. 1–38, 1977.
[2] - J. Lin, “Divergence measures based on the Shannon entropy,” IEEE [2] - J. Lin, “Divergence measures based on the Shannon entropy,” IEEE Trans. Inform. Theory, vol. 37, pp. 145–151, Jan. 1991.Trans. Inform. Theory, vol. 37, pp. 145–151, Jan. 1991.
[3] - C. Stauer and W.E.L. Grimson, “Adaptive background mixture models [3] - C. Stauer and W.E.L. Grimson, “Adaptive background mixture models for real-time tracking," in Proceedings 1999 IEEE Conference on Computer for real-time tracking," in Proceedings 1999 IEEE Conference on Computer Vision and Pattern Recognition, Fort Collins, CO, June 23-25 1999, vol. 2, pp. Vision and Pattern Recognition, Fort Collins, CO, June 23-25 1999, vol. 2, pp. 246-252.246-252.
![Page 39: Two Examples Of Indoor And Outdoor Surveillance Systems: Motivation, Design, And Testing](https://reader036.vdocuments.site/reader036/viewer/2022062518/5681402a550346895dab8b29/html5/thumbnails/39.jpg)
References (cont.)References (cont.)
[4] D. B. Reid, “An algorithm for tracking multiple targets”, IEEE [4] D. B. Reid, “An algorithm for tracking multiple targets”, IEEE Transactions on Automatic Control, vol. 24, pp. 843{854, 1979.Transactions on Automatic Control, vol. 24, pp. 843{854, 1979.
[5] I. J. Cox and S. L. Hingorani, “An efficient implementation of Reid's [5] I. J. Cox and S. L. Hingorani, “An efficient implementation of Reid's multiple hypothesis tracking algorithm and its evaluation for the purpose multiple hypothesis tracking algorithm and its evaluation for the purpose of visual tracking," IEEE Transactions on Pattern Analysis and Machine of visual tracking," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 18, no. 2, pp. 138-150, 1996.Intelligence, vol. 18, no. 2, pp. 138-150, 1996.
[6] L. Lee, R. Romano, and G. Stein, “Monitoring activities from multiple [6] L. Lee, R. Romano, and G. Stein, “Monitoring activities from multiple video streams: Establishing a common coordinate frame," IEEE video streams: Establishing a common coordinate frame," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 22, no. Transactions on Pattern Analysis and Machine Intelligence, vol. 22, no. 8, pp. 758 { 767, 2000.8, pp. 758 { 767, 2000.