NSF Industry-University Cooperative Research Center for Advanced Knowledge Enablement

NOA Inc DBA TerraFly Inc

IBM

Naphtali Rishe

Control and mapping of airborne video sensors

GIS & Data

GIS Analytics GIS & Data for South America

Moving Objects, Modeling Data Transmission

Predictive Spatial Modeling for Diseases

Moving Video Streaming

Aerial Imaging from Balloons

The Challenge

• Securing a vast border region with limited human and capital resources

• Border Control

• Irregular Immigration

• Crime and Illegal Trafficking

• Persistent, live surveillance combined with intelligence and operations is often needed, but not available

Our Solution: CARMEL-TerraFly

Users are able to: • Select a geographic area of interest• Retrieve multimedia data from mobile sensors in the area• View streaming videos in real time (vehicles, people, animals,

etc.)• Easily set temporal, geographic, textual and structural

constraints • Browse relevant video clips synchronized with map display of the

ground projection of the area video-recordedMAP querying and visualization is provided by TerraFly® (http://TerraFly.com)

CARMEL-TerraFly

• Demo Video: http://terrafly.com/CT-video/• Web Simulator: http://terrafly.com/CS/

User Interface

Solid trapezoids are current projections of cameras’ view. The dotted trapezoid is synchronized with the playback of the selected camera.

CARMEL: Geo-monitoring of stationary, mobile and airborne video streams

CARMEL

CARMEL Location TracesSurveillance paths for real-time viewing of streaming videos in the path of interest, perform end-to-end tracking of moving objects and make historical comparisons

CARMEL: Geo-monitoring of desired surveillance paths

Surveillance Path

Area Overview:• Red trace – already traveled• Yellow trace - enroute

CARMEL-TerraFly

Location Trace Monitoring & AlertsQuickly and easily zoom into surveillance areas and paths for close monitoring and review, with data fusion and alert capabilities such as scheduled arrivals and departures.

Time-sensitive and detailed monitoring to help pinpoint what you need to know

Surveillance Path

Scheduled ArrivalExpected Cargo &

Activities

CARMEL

Persistent Monitoring Using Mobile SensorsOur ability to effectively visualize moving objects extends to providing real-time, geolocated video and data streams from mobile sensors, such as drones or boats, and allowing fusion and tracking via multiple moving sensors.

Real-time monitoring of moving objects and use of mobile, geolocated sensors

Real-time, geolocated video surveillance via moving sensors

(e.g., drones, balloons, etc.)

Maintain detailed and aerial overview of the area

of interest

Border Surveillance Scenario

In an area along the El Paso/Mexico border that is known for illegal immigration crossings as well as drug smuggling, a border patrol officer is conducting routine surveillance by viewing a video stream from a drone.

During the surveillance, the officer sees unusual activity on the river’s edge.

Surveillance Path

Border Surveillance Scenario

The officer is able to see what is occurring in the area in real time:• A car parked in an area

on the Mexican side of the border that provides easy access to the river.

• A person running between the river and highway on the US side of the border.

Car parked by the river

Person running to the highway

Border Surveillance Scenario

To determine whether this is an illegal crossing or drug smuggling operation, the officer is able to:• “rewind” surveillance

video of the area to see that it is indeed an illegal crossing of the boarder

• call on-the-ground units for immediate dispatch to the area.

Person running to the river

Border Surveillance Scenario

Because the surveillance is in real time, the officer is also able to keep constant track of the individuals who cross the border illegally to help the on-the-ground officers apprehend the suspect.

Follow the running person

Border Surveillance Scenario

The officer is also able to:• search for historical

videos of the same area to determine if this location has been used in the past for illegal crossings,

• Find a new pathway into the river at this location, which indicates that this is a new illegal crossing point.

New Pathway

Historic Video Current Video