Download - Synesis Embedded Video Analytics
Embedded Video Analytics
DSP Algorithms forDetection, Tracking and Recognition
http://synesis.ru/
Media and Internet
Face detection and recognition servers
Intelligent Video Surveillance
Intelligent cameras, encoders and DVRs
Digital TVDVB receivers,
STBs, PVRs,media centres
HD Intelligent Network Video
Efficient video surveillance (1)
Accurateevent recognition• correct classification• false positives and
false negatives• response time• documentation
?
Efficient video surveillance (2)
Widespread infrastructure• Cross-correlation of
events captured by multiple cameras and other sensors
• Alert prioritization • Distributed attacks
(multiple point intrusions)
Efficient video surveillance (3)
Operator productivity
• Keep attention focused• Reduce subjectivism• Increase response time
Efficient video surveillance (4)
Cost of ownership
• Deployment• Maintenance• Telecom service charges• Minimum team size• Training• Upgrade
(Investment protection)
What is video analytics?
X, Y, Z
Functions of video analytics
1. Anti-tampering and operability monitoring2. Operational alerts
– Automatic priorities
3. Automatic PTZ-camera targeting4. Event recording for instant forensic analysis5. Optimal usage of
network bandwidth and storage memory
Solution: embedded video analytics• Edge device transmits video and
metadata (object and its behaviour description)
VIDEO
METADATAEVENT
DATABASE
Zone 5intrusiondetected
EVENT RULES
Embedded vs server analytics
camera orencoder
video management system or DVR
compressedvideo & audiocodecs video-
analytics
video management system or DVR
ip-cameraor encoder
video and audiocodecs
videoanalytics
metadata
Embedded(front-end)analytics
Server(back-end)analytics
BOTTLENECK
Video signal sources
1. Analoguestandard definition cameras(PAL/NTSC)
2. Network cameras(standard and highdefinition)
3. Thermal cameras
Network cameraAxis 211A
Thermal cameraTitan-14
Wide angle perimeter surveillance(multiple tripwire alert levels)
Fence crossing detector
Apartment housing event recording
Directional detector
Running behaviour recognition
Time-based loitering behaviour recognition
Split target /abandon luggage detection
Group people tracking
Tampering and malfunction detectors• Loss of signal• Obstruction• Out of focus and lens
dusting• Blackout and overexposure • AE failure• Lighting
failure
Upon a suspicious event…• PTZ-targeting• System notification
over IP network to VMS– Sound and visual alarms, SMS etc
• ‘Dry contact’ signal• High quality recording to local
or remote storage (NAS)• Analogue output to legacy
systems (matrix or DVR)
Digital image stabiliser (antishaker)• Eliminates video shaking
caused by wind and industrial vibrations • Essential for analytics performance• Differentiates the camera movements
from scene background/foreground movements
Video analytics components
Detection
Tracking
Recognition
Sterile zone Public spaces
Rare appearance Occasional appearance People flow
perimeter security,strategic
infrastructure
apartment housing, petrol stations, office
buildings
airports,railway stations,
underground
Object tracker complexity
complexity
Dynamic texture of the real world
Dynamic texture modelling
• 4D-pyramid• Feature
probability cloud• α-channel (mask) for
each object
BACKGROUND OBJECT HAAR FEATURES
People group tracking (Q4 2010)
• Feature cloud enables object tracking under partial visibility
• Z-buffer to identify object occlusions
Rule based behaviour recognitionEach zone is configured independently
Zone entrance
Zone exist
Zone loitering:Staying overpredefined period of time
Zone running:Exceeding a predefined speed
Directional move within zone
Metadata sent over IP network / ONVIF• Event type, data and time• Zone or tripwire number• 2D object feature:
– Position, size, area, speed• Real 3D features
– Estimated from 2D featuresusing calibration data
• JPEG frame image withobject trajectory annotation
Videoanalytics calibration
• Two human figures define scale & angle
• Drag’n’drop calibration
• Tracking region
• 2D to 3D coordinate transform
Video analytics parameters1. Service detectors2. Antishaker3. Object tracker
1. Contrast sensitivity2. Special sensitivity3. Min. stabilisation time
4. Object filters1. Maximum object speed2. Min and max areas
1
2
3
4
Video analytics evaluationMethods and results
Video analytics public testsOrganisation Videoanalytics tests
Home Office Scientific Development Branch (HOSDB), UK
• Imagery library for intelligent detection systems (i-LIDS)
National Institute of Standards and Technology (NIST), USA
• AVSS 2009 Multi-Camera Tracking Challenge (based on i-LIDS)
• Face Recognition Vendor Test (FRVT)
Institute of Electrical and Electronics Engineers (IEEE), USA
• Performance Evaluation of Tracking and Surveillance (PETS)
• International Workshop on Performance Evaluation of Tracking and Surveillance
• International Conference on Advanced Video and Signal Based Surveillance
Sterile Zone Performance38 hours, PAL (720 x 576 x 25 fps), M-JPEG, 40 MbpsNumber of true positive alarms: a = 432
False positives alarms (type I error): b = 2
False negatives alarms (type II error): с = 0
Role Recall bias Recall rate Precision Weighted average
Operating alert 0.65 1.00 1.00 1.00
Event recording 75.00 1.00 1.00 1.00
Resolution vs width field of view (FoV)
7-12 m
12-23 m
27-37 m
Maximum response time
• People walking and running–2 seconds
• People moving slowly(e.g. crawling)–10 seconds
Causes of false negatives(simple motion detectors)
• Unstable background decreases sensitivity of an adaptive detector
DYNAMIC TEXTURE MODELING ALGORITHMSENABLE ROBUST OBJECT DETECTION IN A CHALENGING ENVIROMENT
Causes of false positives(basic motion detectors)
• Variable lighting– Shadows from moving clouds and sun– Moving trees, bushes and water
• Camera shaking• Animals, birds and insects• Object trajectory split and double detection• Snow, rain, fog
Examples of false positives(simple motion detectors)
INSECT RABBIT
CAMERA SHAKING
VIDEO ANALYTICS PREVENTS FALSE ALARMS CAUSED BY THESE FACTORS
BIRD
Object trackingwhilst tree shadows moving
Performance estimation by3D security modeling
• 3D modeling– building infrastructure– control zones of cameras
and third-party detectors– treats (in space-time)
• Estimation of detection probabilities under variable external conditions– day/night, fog, snow
• Video presentation
ORIGINAL BUILDING
3D MODEL OF BUILDNG
Hardware reference designsMultifunctional video services and HD cameras
with embedded analytics
System-on-chip video analytics
Videoanalytics
HD H.264 codec
Linux Videofilters
1080p
Dual channel video analytics encoder
04/11/2023 44
• Two analogue inputs (BNC)• Two managed outputs (BNC)
and digital video over IP• H.264 & MJPEG encoding• Embedded video & audio analytics• POE+ and backup power• ONVIF 1.01 support• - 40⁰...+50⁰ С• Lightning guard
ANALOG + IPHYBRID TECHNOLOGY
Dual channel video analytics encoder
Interfaces
LAN USB I/OAUDIO OUT
POWER BATTERY
AUDIO INRESET
HD video analytics camera
MJPEG vs H.264 compression
HD 1080i HD 720p D1 480p0
5
10
15
20
25
30
35
40
H.264MJPEG
DA
TAF
LO
W, M
BP
S
RESOLUTION
H.264 MJPEGHD 1080i 2.3 34.1HD 720p 1.8 19.6D1 480p 1.5 3.4
Applications and use-casesVideo analytics encoder
Self-contained intelligence for perimeter security
Integrated solution:1. Embedded video analytics2. Automatic PTZ targeting3. Unlimited, multizone sensor
integration (I/O, RS485)4. Active illumination5. Two-way intercom6. Backup power &
battery management MB
Sophisticated landscape
Strategic infrastructure
Cost-effective upgrade oflegacy analogue infrastructure
• No cable or camera replacement required• Increase storage efficiency by 10-100 times• Automatic operational alerts• Intelligent search using recorder events• Future proof network surveillance via ONVIF
Local/backup storage• Detachable video storage
– USB 2.5” hard drive or flash memory• Accurate timestamp (NTP sync)• Backup storage if NAS not available• Portable player, video can be played on any PC
Unique selling position1. Fully embedded (DSP) implementation
– Real-time processing of uncompressed video– HD/Megapixel resolution– Highly scalable
2. Unmatched performance in harsh environment– dynamic texture engine
3. Wide interoperability– ONVIF compliance
Example of customization
1. Custom user interface2. Custom network and serial protocols3. Overlay text (POS, industrial etc)4. Custom DaVinci codecs (e.g. H.264 SVC)5. Custom video analytics
Future of video surveillanceMultiple camera tracking using 3D model
Segmentation problemand object occlusions
‘Single camera’video analytics
AB
C
A
‘Multiple camera’video analytics
i-LIDS multiple camera tracking scenario
2 3 4
11/04/2023 www.synesis.ru 60
1 2
Камера 1 Камера 2
3D model of a buildingand camera controlzones
Video analytics + 3D modeling
11/04/2023 61
OBJECT UNIQUE ID PRESERVED WHEN TRACKING FROM CAMERA TO CAMERA
3D trajectory reconstructed frommultiple video sources