moving object detection on a runway prior to landing using an onboard infrared camera

MOVING OBJECT DETECTION ON A RUNWAY PRIOR TO LANDING USING

AN ONBOARD INFRARED CAMERA

Dr. Gerard Medioni

Cheng Hua Pai

Yu Ping Lin

Introduction

Input: Infrared runway sequence

Goal: Detect moving objects on runway

Approach

We do it in two steps:1. Stabilize the sequence

2. Detect motion on the stabilized sequence

Flow chart of the system

Runway Identification

Image Stabilization

Motion Detection

irefH ,

Blobs in motionUpdate Referenceframe?

Update Referenceframe

Yes

ReferenceFrame

Stabilization

Issues: Planar region containing the runway Feature choice and matching Transformation between consecutive frames

Stabilization

Approach Manually Label

planar region SIFT provides

sufficient and descriptive features

RANSAC to estimate best transformation

Stabilization

Result:

Stabilized runway sequence

Adaptive Reference Frame

Issues: For longer sequence

Small errors accumulate Big scale difference

Beginning of a Sequence End of a Sequence

Adaptive Reference Frame

When to change reference frame? Check the lower edge length ratio

Stabilization algorithm

Landing UAV image sequence

Manually labeled planar region

3. Match features to previous frame to establish correspondence

1. Extract SIFT features

2. Region of Interest

4. Use RANSAC to remove outliers and estimate homography

6. Warp to the reference frame

5. Update reference frame if necessary

input

output

Locally stabilized image sequence and for all sirefiH ,

Adaptive Reference Frame

Result:

Original Sequence Locally Stabilized Sequence

Detection module

Issues: Detection method Global intensity variation Noise

Moire in the sequence Poor stabilization Local intensity variation Random noise

Detection

Approach: Use simple Gaussian background model

t = (1-) * (t-1) + * (It)t2 = (1-) * (t-1) 2 + * (It- t)

Foreground: More than 4t2 from mean

t

4t2 4t

2

Foreground Background

Intensity distribution of an image

Global intensity variation

Approach: Compensate gain with affine transformation

[Yalcin 05]

Before compensation After compensation

ttttt bImI 1

Noise reduction

Approach: Moire in the sequence

Compare 8 neighbouring background pixels Poor stabilization

Restabilize with gradient map (also SIFT)

To Gradient

Noise reduction

Approach: Local intensity variation

Intensity normalization on the foreground pixels

Random noise Compare consecutive foreground masks

iifg

iifg

i bu= mI

With random noise Without random noise

Detailed flow chart of Motion Detection Module

Locally Stablized Runway

Sequence

Reference Frame update?

Filter Runway

Intensity Normalization

Intensity compensated runway image

Image Subtraction

Foreground mask & Quality

Indicator

QI. Score

Noise reduction

Update Runway Filter

Runway Filter

Update Reference Frame

Reference Frame

Update Background Model

Background model

Foreground mask

Yes

No

Good

Bad

Motion Detection Module

Homographies Hi,ref

Detection Result

Result:

Locally Stabilized Sequence Foreground mask

Evaluation

Tested on 150 synthesized and 18 real-world sequences

Results (synthetic data):

Obj. size

Conclusion

Detection affected by: Object speed and size Threshold parameters

Program limitation: Moving objects fade in and out Bad result near the end of the sequence

Future work: More test on larger dataset Speed improvement

Reference W. G. Chris Stauffer. Adaptive background mixture models for real-time tracking. 1999 IEEE

Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'99), 2:2246, 1999.

M. Fischler and R. Bolles. Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, 24(6):381-395, 1981.

C. Harris and M. Stephens. A combined corner and edge detector. Proceedings of The Fourth Alvey Vision Conference, (1):147-152, 1988.

T. G. R. Kasturi, O. Camps and S. Devadiga. Detection of obstacles on runway using ego-motion compensation and tracking of signicant features. Proceedings 3rd IEEE Workshop on Applications of Computer Vision, 1996 (WACV'96), pages 168-173, 1996.

D. G. Lowe. Distinctive image features from scale invariant keypoints. International Journal of Computer Vision, 60(2):91-110, 2004.

R. S. B. Sridhar and B. Hussien. Passive range estimation for rotor-craft low-altitude flight. Machine Vision and Applications, 6(1):10-24, 1993.

S. Sull and B. Sridhar. Runway obstacle detection by controlled spatiotemporal image Low disparity. IEEE Transactions on Robotics and Automation, 15(3):537-547, 1999.

R. C. H. Yalcin and M. Hebert. Background estimation under rapid gain change in thermal imagery. Second IEEE Workshop on Object Tracking and Classification in and Beyond the Visible Spectrum (OTCBVS'05), 2005.

Q. Zheng and R. Chellappa. Motion detection in image sequences acquired from a moving platform. Proc. Int. Conf. Acoustics, Speech, and Signal Processing, Minneapolis, 5:201-204, 1993.