astrophysics applied what to do with images and wavelets plato's republic, book vii: socrates:...

Astrophysics Applied What to do with Images and

Wavelets

Plato's Republic, Book vii:

Socrates: Shall we set down astronomy among the subjects of study?

Glaucon: I think so, to know something about the seasons, the months and the years is of use for military purposes, as well as for agriculture and for navigation.

Socrates: It amuses me to see how afraid you are, lest the people should accuse you of recommending useless studies.

Socrates goes on to say that the use of astronomy is not to add to the vulgar comforts of life, but to assist in raising the mind to the contemplation of things which are to be perceived by the pure intellect alone.

28th. June 2006 Bernard60 - Valencia, Spain 2

What this is about – a history

1984: Started a small hi-tech venture – to offset the stress of the academic job situation (!). The venture focused on CCTV imaging on the then-new PC platform.

1988: BBC TV Tomorrow’s World

1995 started work in earnest after winning UK government contracts

2003 developed leading video acquisition, storage and retrieval system based on wavelet methodology, modern statistical methods and nonlinear image processing algorithms.

Currently happily spend a week per month at the Kapteyn Institute in Groningen: mind-to-mind resuscitation.


Achievements

Best real-time video compression algorithm based on adaptive wavelet technology: up to 9 days on one DVD!

Sophisticated video event detection through adaptive scene analysis.

Automated video synopses based on event detection allows searching terra-bytes in minutes

Earning a reasonable living without having to suffer the slings and arrows of arbitrary high level funding decisions – independence!!

Even more stress and anxiety


Image processing

Static images

Weiner filters

Fourier transforms

Moving image sequences

Nonlinear reconstruction

Wavelet transforms

Classical Modern

Static images (generally)

Huge dynamic range

Measured quantities have important physical meaning

Dynamic images

Limited dynamic range

Data is not photometric

TV ImagingAstronomy


CCTV – Closed circuit television

The UK has some 8x106 CCTV cameras– 10% of world total– Main detector is small CCD (mostly colour)– Sensitivity down to 10-5 lux (mono cameras)– Resolution at best 576x704– Output is analogue

Future is to go to HDTV and digital IP delivery– Resolution 1024x768 (p) or 1280x1024 (i)– Data format MPEG-4 (ideally AVC – level 10)

starlight 0.00005 lux

Moonlight 1 lux

Office light 100 lux

midday 100,000 lux

(1 footcandle = 10.76 lux)

Note that even the new HDTV has nowhere near the 5 megapixel resolution of a modest still digital camera.


The CCTV data nightmare

One camera generates 30 Megabytes of data per second.

A small 100 camera system delivers 3 Gigabytes per second or 1 Petabyte per year

All CCTV cameras in the UK generate about 100 Exabytes per year

There is no way of storing all this, let alone looking at it or searching for anything.

G=109, T=1012,P=1015, E=1018,Z=1021,…,W=1030

Library of Congress = 10 Tbytes

Cut down on frame rate (time-lapse)

Save only 1-2 weeks

Compress while trying to retain quality

Save only “interesting” data

Such strategies may compromise data integrity if data is eventually to be used as part of a legal investigation.

Some numbers … Traditional strategies for handling this …

While acquiring the images

1. Do far better compression

2. Analyze the data

3. Enhanced real time scene analysis

4. Store data synopses

Improved search methodology:

1. Cross-correlate what is seen on different cameras

2. Learn from previous searches

3. Provide a search query mechanism


Acquiring video pictures to PC

The video frame grabber converts analogue video signals into digital data array that can be processed. Much depends on the quality of this device: this one supports broadcast quality.

Analogue video in (16 channels)

Digital data outputSent to PC processor

Analogue to digital converters

High performance digital signal

processor (VLIW multi-core

4 billion instructions per second)

One or more of these sit inside a

“PC” running Linux

We design and manufacture

these.This is the 6th in 20

years.


Typical site monitoring


One dimensional wavelet filter

)(][

)(][)(

,

,

xkd

xksxf

kLjk

LjjL

kjk

j

The yellow bits at level j

Scaling function at level j

(orthogonal to the wavelet)

The red bits. Contributions come from all

levels, L

Wavelet Function at level L

Reconstruction of data S0 from multi-resolution analysis {Sj, Dj, Dj-

1}.

Schematically:

Levels used in reconstruction

You don’t have to use the same wavelets at all levels, and nor do you have to keep all the {dj}.

The choices can be made to depend on image content and dynamics. This is a key trick.


Multi-resolution hierarchy

2D S

2SS 2SD

Level 1

Level 2

Level 3

1 SS

2D D

1D S 1 D D

1SD

O riginal Im ageLevel 0

STAR T

FINISH

1D D1D S

1S D

2SD

2DS 2DD

3SS 3SD

3D S 3D D

The image is shrunk by linear factors of two. The residuals that would allow the picture at any level to be reconstructed from the higher levels are stored alongside.

The shrinking process is important, but need not use wavelets.


Image representation - wavelets

Two level wavelet transform

Censorship of coefficients for compression

Rie

n va

n de

Wey

gaer

t

Images have been enhanced for clarity

The scene analysis might as well be done at the same time as the censorship and compression of the data. That way the compression can adapt locally to scene content.


Wavelet compression vs. MPEG

Wavelet compressed frame from CCTV video sequence.

MPEG-1 compressed frame from same video sequence. Note the blocky artefacts and colour contouring.

The goal is to achieve high levels of compression without sacrificing image quality. This is particularly difficult when there is a lot of movement in the scene. Using our locally adaptive wavelets makes a huge difference over normal wavelets.


Sunlight, shadows & trees

This is full-screen video event detection. No areas have been excluded.

Note the moving shadows and trees, together with varying illumination. These do not produce alarms.

The first detection is in fact behind the window before walking through the door.

The picture turns grey on detection for demo purposes only: we need to show colour blocks.

This is an MPEG-1 version of an Astraguard wavelet sequence

The main issue is to detect what you want to detect without missing anything, while at the same time avoiding things you do not want like sunlight, shadows and trees. This sequence demonstrates some successes. There are also some failures which can in fact be handled with a little more algorithm refinement.


Application Areas

Large government and industrial sites

Hospitals

Police Work

Fire detection on gas rigs

Oil Pipelines

Power line monitoring

High value premises

The goal is to provide equipment to handle video streams in situations where an understanding of what is taking place in the scene, or has taken place at an earlier time, is relevant.

The system must be easy to configure and even easier to use. Application areas are often mission-critical, so close to 100% reliability is essential.

Systems range from one camera to many hundreds of cameras. Upper limit is in the 1000’s of cameras.


What next?

Retire to a vineyard in Australia?

Write the book that has taken over 10 years to not finish.

Continue to share in the excitement of cosmology.

Keep healthy if not fit


Signal recognition using wavelets

The same signal (top) is analyzed using two different wavelets.The Gabor wavelet is capable of localizing the signal in both frequency and

time.

“fre

qu

en

cy”

Time

am

plit

ud

e


Signal recognition using wavelets

“fre

qu

en

cy”

Time

am

plit

ud

e

The same signal (top) is analyzed using two different wavelets.The Gabor wavelet is capable of localizing the signal in both frequency and

time.

astrophysics applied what to do with images and wavelets plato's republic, book vii: socrates:...

Documents