The work has been supported by the FastPass project. The research leading to these results has received funding from the European Union Seventh Framework Programme (FP7/2007-2013) under grant agreement n° 312583. This publication only reflects the author’s view and the European Union is not liable for any use that may be made of the

information contained therein. All document contained therein cannot be copied, reproduced or modified in the whole or in the part for any purpose without written permission from the FastPass Coordinator with acceptance of the Project Consortium.

Visual Surveillance Technologies for Enhancing ABC Secure Zones

Project: FastPassCsaba Beleznai, Stephan Veigl, Michael RauterDavid Schreiber, Andreas Kriechbaum

Video- and Security TechnologySafety & Security DepartmentAIT Austrian Institute of Technology GmbH

The work has been supported by the FastPass project. The research leading to these results has received funding from the European Union Seventh Framework Programme (FP7/2007-2013) under grant agreement n° 312583. This publication only reflects the author’s view and the European Union is not liable for any use that may be made of the information contained therein. All document contained therein cannot be copied, reproduced or modified in

the whole or in the part for any purpose without written permission from the FastPass Coordinator with acceptance of the Project Consortium.

2

Motivation

Significantly increasing passenger flows• from 2012 to 2016 +800 million

Border guards face big challenges• in-depth document checks• reliable identity checks• check of entry conditions• discover possible threats

The work has been supported by the FastPass project. The research leading to these results has received funding from the European Union Seventh Framework Programme (FP7/2007-2013) under grant agreement n° 312583. This publication only reflects the author’s view and the European Union is not liable for any use that may be made of the information contained therein. All document contained therein cannot be copied, reproduced or modified in

the whole or in the part for any purpose without written permission from the FastPass Coordinator with acceptance of the Project Consortium.

3

Motivation

Issues to be addressed Exactly one person per passport

Single person detection

Clean secure zone Left object detection

Situation overview Queue length estimation

Video Surveillance in Gate Reliable detections with 3D

stereo video cameraDemo eGate: Vienna Airport(Terminal 2, Non-Schengen-Arrivals)

The work has been supported by the FastPass project. The research leading to these results has received funding from the European Union Seventh Framework Programme (FP7/2007-2013) under grant agreement n° 312583. This publication only reflects the author’s view and the European Union is not liable for any use that may be made of the information contained therein. All document contained therein cannot be copied, reproduced or modified in

the whole or in the part for any purpose without written permission from the FastPass Coordinator with acceptance of the Project Consortium.

ABC Gates

4

The work has been supported by the FastPass project. The research leading to these results has received funding from the European Union Seventh Framework Programme (FP7/2007-2013) under grant agreement n° 312583. This publication only reflects the author’s view and the European Union is not liable for any use that may be made of the information contained therein. All document contained therein cannot be copied, reproduced or modified in

the whole or in the part for any purpose without written permission from the FastPass Coordinator with acceptance of the Project Consortium.

5

The Sensor

3D stereo-camera system developed by AIT Area-based, local-optimizing, correlation-

based stereo matching algorithm Specialized variant of the Census Transform Video image

• Spatial Resolution: 752x480 rectified to 608x328

• RGB Depth information

• 16 bit depth image USB 2 interface

The work has been supported by the FastPass project. The research leading to these results has received funding from the European Union Seventh Framework Programme (FP7/2007-2013) under grant agreement n° 312583. This publication only reflects the author’s view and the European Union is not liable for any use that may be made of the information contained therein. All document contained therein cannot be copied, reproduced or modified in

the whole or in the part for any purpose without written permission from the FastPass Coordinator with acceptance of the Project Consortium.

6

Camera Set-Up (inside the gate)

Top-view of eGate interior ~15 frames per second Mounting height: 279 cm depth resolution: ~2 cm Ground floor 180x60 cm spatial resolution: ~1 cm

The work has been supported by the FastPass project. The research leading to these results has received funding from the European Union Seventh Framework Programme (FP7/2007-2013) under grant agreement n° 312583. This publication only reflects the author’s view and the European Union is not liable for any use that may be made of the information contained therein. All document contained therein cannot be copied, reproduced or modified in

the whole or in the part for any purpose without written permission from the FastPass Coordinator with acceptance of the Project Consortium.

7

Single Person Detection

Motivation / Challenges Ensure only one person is inside the eGate

• reliable detection and counting of persons• multiple persons must not pass!

Real-time processing• low latency

Better than existing system• error-prone to tailgating / piggybacking• number of false alarms

Developed for secure zones

The work has been supported by the FastPass project. The research leading to these results has received funding from the European Union Seventh Framework Programme (FP7/2007-2013) under grant agreement n° 312583. This publication only reflects the author’s view and the European Union is not liable for any use that may be made of the information contained therein. All document contained therein cannot be copied, reproduced or modified in

the whole or in the part for any purpose without written permission from the FastPass Coordinator with acceptance of the Project Consortium.

8

Single Person Detection

Method1. Head candidates

• hypothesis generation test positions

2. Classification with head-shoulder model• support-vector-machine

filter false hypothesis

3. Tracking of detections use door signals to increase

robustness

The work has been supported by the FastPass project. The research leading to these results has received funding from the European Union Seventh Framework Programme (FP7/2007-2013) under grant agreement n° 312583. This publication only reflects the author’s view and the European Union is not liable for any use that may be made of the information contained therein. All document contained therein cannot be copied, reproduced or modified in

the whole or in the part for any purpose without written permission from the FastPass Coordinator with acceptance of the Project Consortium.

Single Person Detection

Examples - single person

9video 1 - single person.avi

The work has been supported by the FastPass project. The research leading to these results has received funding from the European Union Seventh Framework Programme (FP7/2007-2013) under grant agreement n° 312583. This publication only reflects the author’s view and the European Union is not liable for any use that may be made of the information contained therein. All document contained therein cannot be copied, reproduced or modified in

the whole or in the part for any purpose without written permission from the FastPass Coordinator with acceptance of the Project Consortium.

Single Person Detection

Examples - multiple persons

10

video 2 - multiple persons.avi

The work has been supported by the FastPass project. The research leading to these results has received funding from the European Union Seventh Framework Programme (FP7/2007-2013) under grant agreement n° 312583. This publication only reflects the author’s view and the European Union is not liable for any use that may be made of the information contained therein. All document contained therein cannot be copied, reproduced or modified in

the whole or in the part for any purpose without written permission from the FastPass Coordinator with acceptance of the Project Consortium.

Single Person Detection

Examples - closed doors

11

video 3 - closed doors.avi

The work has been supported by the FastPass project. The research leading to these results has received funding from the European Union Seventh Framework Programme (FP7/2007-2013) under grant agreement n° 312583. This publication only reflects the author’s view and the European Union is not liable for any use that may be made of the information contained therein. All document contained therein cannot be copied, reproduced or modified in

the whole or in the part for any purpose without written permission from the FastPass Coordinator with acceptance of the Project Consortium.

12

Left Object Detection

Motivation / Challenges Detect left objects in eGate

• eGate has to be empty after person left• visualize left object

Real-time processing• low latency

Difficult object size and / or apperance• small size (e.g. passport)• low contrast (e.g. empty bottle)

The work has been supported by the FastPass project. The research leading to these results has received funding from the European Union Seventh Framework Programme (FP7/2007-2013) under grant agreement n° 312583. This publication only reflects the author’s view and the European Union is not liable for any use that may be made of the information contained therein. All document contained therein cannot be copied, reproduced or modified in

the whole or in the part for any purpose without written permission from the FastPass Coordinator with acceptance of the Project Consortium.

13

Left Object Detection

Method 3D Images

• color image• depth information• data fusion

Motion suppression

The work has been supported by the FastPass project. The research leading to these results has received funding from the European Union Seventh Framework Programme (FP7/2007-2013) under grant agreement n° 312583. This publication only reflects the author’s view and the European Union is not liable for any use that may be made of the information contained therein. All document contained therein cannot be copied, reproduced or modified in

the whole or in the part for any purpose without written permission from the FastPass Coordinator with acceptance of the Project Consortium.

Left Object Detection

Examples - Trolley

14

video 4 - trolley.avi

The work has been supported by the FastPass project. The research leading to these results has received funding from the European Union Seventh Framework Programme (FP7/2007-2013) under grant agreement n° 312583. This publication only reflects the author’s view and the European Union is not liable for any use that may be made of the information contained therein. All document contained therein cannot be copied, reproduced or modified in

the whole or in the part for any purpose without written permission from the FastPass Coordinator with acceptance of the Project Consortium.

Left Object Detection

Examples - Mobile Phone

15

video 5 – mobile phone.avi

The work has been supported by the FastPass project. The research leading to these results has received funding from the European Union Seventh Framework Programme (FP7/2007-2013) under grant agreement n° 312583. This publication only reflects the author’s view and the European Union is not liable for any use that may be made of the information contained therein. All document contained therein cannot be copied, reproduced or modified in

the whole or in the part for any purpose without written permission from the FastPass Coordinator with acceptance of the Project Consortium.

Left Object Detection

Examples - Trolley

16

video 6 – bottle.avi

The work has been supported by the FastPass project. The research leading to these results has received funding from the European Union Seventh Framework Programme (FP7/2007-2013) under grant agreement n° 312583. This publication only reflects the author’s view and the European Union is not liable for any use that may be made of the information contained therein. All document contained therein cannot be copied, reproduced or modified in

the whole or in the part for any purpose without written permission from the FastPass Coordinator with acceptance of the Project Consortium.

17

Queue Length Estimation

Motivation / Challenges Enhanced queue management

• number of persons per queue

Visual tracking of queue dynamics• estimated waiting time

Overcome occlusion problems• eliminate top-view requirement

(for low ceiling height environments)• use 3D information

4 Min.

The work has been supported by the FastPass project. The research leading to these results has received funding from the European Union Seventh Framework Programme (FP7/2007-2013) under grant agreement n° 312583. This publication only reflects the author’s view and the European Union is not liable for any use that may be made of the information contained therein. All document contained therein cannot be copied, reproduced or modified in

the whole or in the part for any purpose without written permission from the FastPass Coordinator with acceptance of the Project Consortium.

18

Queue Length Estimation

Outlook (Method) Stereo camera (e.g. mounted on eGate) Detection distance up to 12 meters

The work has been supported by the FastPass project. The research leading to these results has received funding from the European Union Seventh Framework Programme (FP7/2007-2013) under grant agreement n° 312583. This publication only reflects the author’s view and the European Union is not liable for any use that may be made of the information contained therein. All document contained therein cannot be copied, reproduced or modified in

the whole or in the part for any purpose without written permission from the FastPass Coordinator with acceptance of the Project Consortium.

Conclusion

Stereo Video Surviallance New dimension of information

Robust and reliable detection under variable situations

Decrease false alarms for• single person detection• left object detection

New options for queue length estimation

19

The work has been supported by the FastPass project. The research leading to these results has received funding from the European Union Seventh Framework Programme (FP7/2007-2013) under grant agreement n° 312583. This publication only reflects the author’s view and the European Union is not liable for any use that may be made of the information contained therein. All document contained therein cannot be copied, reproduced or modified in

the whole or in the part for any purpose without written permission from the FastPass Coordinator with acceptance of the Project Consortium.

Stephan VeiglVideo- and Security TechnologySafety & Security DepartmentAIT Austrian Institute of Technology GmbHstephan.veigl@ait.ac.at

20

AIT Austrian Institute of Technologyyour ingenious partner