On-Demand Processing of UAV Full Motion Video

Using SensorML

Dr. Mike BottsBotts Innovative Research, Inc.

Rocket City Geospatial ConferenceNovember 2010

Introduction

• SensorML is one of the standards within the Open Geospatial Consortium (OGC) Sensor Web Enablement (SWE) framework

• Sensors and sensor data need to become a more readily accessible part of geospatial systems

• SWE provides support web-based support for– Discovery of sensors and observations– Tasking of sensors– Access to observations and alerts (archived and real-time)– Processing – Full characterization of sensor and processing systems

• SWE can and has been applied to new and existing legacy systems

SensorML

• XML-based encoding• Everything is modeled as a process

– Sensors, actuators, platforms– Models, simulators– Processing chains, workflows– QC tests, error propagation, human analysts– Human-machine interfaces

• Provides– Information for discovery (i.e. electronic spreadsheet) for all processes– Lineage chain (i.e. how did this observation come to be? What

processes and testing were involved?)– On-demand , reconfigurable processing (i.e. What further info can I

produce with this data?)

SensorML

UAV Full Motion Video Example

SensorML-enabled Client

Stylers

Tigershark OGC SOS

Video (JP2)

Navigation

Data

The Space Time Toolkit client requests UAV location and attitude data from an OGC Sensor Observation Service (SOS), as well as video, geolocates data on-demand using SensorML, and then renders data according to stylers (e.g. Texture-map styler).

UAV On-Demand GeolocationVideo Clip

• Play clip

• On demand geolocation of Tigershark UAV video using SensorML at Empire Challenge 2008. The video and UAV navigation information (latitude, longitude, altitude, true heading, pitch, and roll) are being served from an SOS; geolocation of footprints and video are being accomplished on-the-fly within the client using SensorML. The SensorML process chain ingest video and navigation data from the SOS and using a sensor model for the video camera, geolocates the video data to the 3D terrain. During this demo, the SOS providing the video imagery and navigation data was in the US (Virginia) while the client was accessing and geolocating the video in Toulouse, France.

• Part 1 - http://www.botts-inc.net/downloads/videos/STT_Tigershark_OnTheFlyGeolocation3_MPEG4.avi

• Part 2 - http://www.botts-inc.net/downloads/videos/STT_Tigershark_OnTheFlyGeolocation4_MPEG4.avi

Other Examples of SWE and SensorML

Conclusions

• SensorML encodings of processes are execution-engine agnostic– Open Source execution engine exists

• SensorML processes can be discovered, downloaded and uploaded, reconfigured, and executed in real-time

• SensorML processes can be executed wherever it makes the most sense– at a data center, at a web service, in the client (exploitation tool or on soldier’s arm), or at

the sensor system

• SensorML process chains can include and mix processes that are implemented locally with those implemented on web services

• Can choose to take the process to the data rather than bringing the data to the process

Links

• Botts Inc – http://www.botts-inc.net• More videos – http://www.botts-inc.net/videos.html • Overview presentations - http://www.botts-inc.net/presentations.html

• Sensor Web Enablement – http://www.ogcnetwork.net/SWE • SensorML - http://www.ogcnetwork.net/SensorML

• Open Geospatial Consortium (OGC) - http://www.opengeospatial.org/