recent advances in viper david mihalcik david doermann charles lin

Recent Advances in ViPER

David Mihalcik

David Doermann

Charles Lin

What is ViPER?

A tool for evaluating video understanding algorithms.

Includes:– An Annotation Tool

For labeling ground truth and browsing results.

– A Comparison Tool For evaluating result data with respect to ground truth.

What is the Problem?

Lots of people, here and elsewhere, are working on video processing algorithms for information extraction, etc.

Evaluating performance of the algorithms requires a lot of work, usually with tools developed by the algorithm designer.

Is your solution any good? Prove it.

How to evaluate the algorithm?

What is the problem?– ViPER focuses on evaluating solutions to detection

and tracking problems; these determine if or where in the video some entity or event appears.

Evaluation– Comparison of the result data set against a truth

data set.– Truth, metrics, and rules for comparison are task

dependent.

Goal of the ViPER Project

To make evaluation of video algorithms simple, repeatable, and ubiquitous.

As ground truth is required for evaluation, annotation must be made simple, as well.– Avoid tedium.– Avoid frustration.– Support expert usage.

Ground Truth Annotation

ViPER-GT supports annotation of temporally qualified spatial and nominal data on video files and still images.

Go from the simple per-frame or shot annotations to detailed spatial markup.– You can quickly indicate which frames contain people.– Then, you add how many people per frame.– With a lot of time and money, you can put boxes around

them.

ViPER-GT: Video Ground Truth Annotation Tool

For Example: Person Tracking

How well does an algorithm find and track humans moving through a video?

To evaluate detection:– Truth must indicate which frames contain the

person.

To evaluate tracking:– Truth must contain spatial information, indicating

where a person may be found.

Example of Annotation: Person Detection

Example of Annotation: Person Tracking

The ViPER Data Model

Similar to a relational database:– Tables are Descriptor Definitions.– Columns are Attributes.– Rows are Descriptor Instances.

Most descriptors are OBJECT descriptors:– Attributes are temporally qualified.– Static OBJECTS have a frame range, but their

attributes are not temporally qualified. Useful for events, etc.

The ViPER File Format and API

Uses XML. I won’t go into it here. There is a Java API.

Related Work

VideoAnnEx OntoLog PhotoStuff Informedia

VideoAnnEx

IBM’s MPEG-7 annotation tool. Cool Features:

– Cut detection makes it easy to add per-shot markup.– Supports MPEG-7.

Annoyances:– Not very good for spatial attributes.– Commercial software. Not as extensible as ViPER.

OntoLog

Jon Heggland’s Tool for Temporal Markup with Ontologies

Advantages:– Good support for key bindings and playback.– Data model supports inheritance.

Annoyances:– No spatial data support.

PhotoStuff

MINDSWAP’s Tool for Adding Semantic Web Markup to Images

Cool Features:– Semantic Markup!– Spatial Data!

Annoyances:– Buggy and beta.– No support for video.

Informedia

CMU’s tool for browsing video libraries Cool Features:

– Advanced browsing functionality. But…

– Focus on video library, not annotation.– Not available for download, from what I can tell.– Not terrifically extensible.

See also: Silver and Malach

Extending the Interface

ViPER provides a lot of functionality, but is very general.

It may be appropriate to extend viper-gt to better support marking up a different type of annotation.

Example Extension: Adding Text Zones

Adds a toolbar that allows typed bounding boxes.

Instead of having to click create, auto-creates a new box.

Architecture of ViPER-GT

Application Launcher– Loads a set of javabeans from an RDF model.– Allows modifying menus, i18n, etc.– Is a bit of a pain to handle 'menu change' events,

like for most recently used menu. Viper View Mediator

– Javabean container for ViPER API.– Adds 'user interaction' methods to keep track of

things not in API (focus, filters).

Continual Improvement

SourceForge web site:– http://viper-toolkit.sf.net/

Mail suggestions/comments to:– viper@cfar.umd.edu