A research strategic plan for computer vision at NICTA

Market Drivers

• Safety and Security:– SAFE: est video surveillance software in US – $1.8-4

billion

• Intelligent Transportation Systems:– STARSense:

• SCATS installed in 1300 cities in 20 countries• More than 9 million vehicle trips daily in Sydney Urban

corridor• Aiming for 5% improvement in traffic flow• Dramatic decrease in cost of traffic sensing deployment

– Automap• Map development cost for TeleAtlas and Navteq est > US$1

billion• Potential price for fresh data for Aust capitals is $10m p/a

Market Drivers (cont)

• Biomedical and Life Sciences– Bionic eye and low vision devices:

• 3,300-4,000 people / million legally blind (AMD, retinopathy, cataract)

• 480,000 people in Australia with visual impairment• Cost of cochlea implant in US > US$60,000

• Environmental Management– GREEnMan:

• Agricultural industries in Australia approximately 135,000 farms

• 65,000 crop or horticultural• Plant industries value $15 billion at farm gate – contribution

to the economy of $10 billion

Problems

Problems

Problems

Problems

Research Approach

• Fusing key theory from– Multi-view Geometry– statistical pattern recognition

• Into key algorithms such as:– Detection– Tracking– Recognition

• To introduce novel theory driven approaches, that provide robust solutions to complex dynamic scene analysis

• Delivered as a NICTA-wide platform for computer vision

Approaches – Building in geometry: Night-time Traffic Surveillance• Objectives

– Count the numbers of vehicles for each lane

– Estimate the speed of the vehicles

– Classify the vehicles as car / truck or bus

– Detect the changes of lane– Make the installation easy,

automatic– Avoid the manual thresholds

(scene dependent)

Approaches – Bionic eye – a running example

Camera+processor Wireless link Implant

Approaches – Bionic eye – a running example

VIsion processing for the Bionic Eye

Signals on electrodes

Dynamic scene understanding:Real-time structure and motion recovery

object identification

Reprojection to RGC array

Approaches – Bionic eye (cont)

• Based on visual processing, context, motion and user feedback, we may choose what information to present at each retinal ganglion cell in each cycle.

Approaches – Bionic eye (cont)

• Centre for Eye Research Australia clinicians running focus groups with low vision subjects (Results June)

• Identified ‘friend recogniser’ as possible demonstrator– From wearable computing– Identify 20 close ‘friends’ in close proximity– Output via audio

• Problem: from wearable cameras/computers– Pedestrian detection/tracking– Face detection/tracking– Face recognition from small database

Approaches – Hardware enhanced detection

Towards Safer Roads by Integration of Road Scene Monitoring and Vehicle Control, L Petersson, L Fletcher, et. al. The International Journal of Robotics Research, 2006

Approaches – pedestrian detection

Sakrapee Paisitkriangkrai, Chunhua Shen, and Jian Zhang. An experimental evaluation of local features for pedestrian classification in DICTA'07, Dec 2007. IEEE Computer Society [Best Paper Award].

Approaches – fusing statistical approaches to tracking

Kernel-based tracking from a probabilistic viewpoint, Quang Nguyen, Antonio Robles-Kelly, Chunhua Shen, IEEE CVPR'07. Minnesota, USA, June, 2007.

Face detection and recognition

Approaches – Multi-body multi-motion recovery

Hongdong Li, Two-view Motion Segmentation from Linear Programming Relaxation,  in Proceedings of CVPR, 2007

Approaches – Spherical optical flow time to collision – real-time motion understanding

C.MacCarthy, N. Barnes and R. Mahony, “A Robust Docking Strategy for a Mobile Robot using Flow Field Divergence”, IEEE Trans Robotics, in press 2008.C. McMcarthy, N. Barnes and M. Srinivasan"Real Time Biologically-Inspired Depth Maps from Spherical Flow", Proc. IEEE ICRA2007, Rome, Italy, May, 2007.

Approaches – Spherical egomotion recovery – statistical Geometry, for robust parallel implementation

John Lim and Nick Barnes, “Directions of Egomotion from Antipodal Points”, accepted, IEEE-CVPR, 2008

Approaches and complexities

NICTA-wide computer vision platform

• Parallel hardware:– Nvidia CUDA GPU– Blackfin DSP– Spartan 3 FPGA– Regular Intel SIMD multicore

• Software– implementations on NICTA Intel PXA270 Microprocessor-

based embedded platform

NICTA-wide platform project

Projects to be proposed

• VIBE (Vision Processing for the Bionic Eye)• Vision platform:

– Dynamic Scene understanding– Parallel embedded platform QRL + CRL– Serial embedded platform– Code contributions from all Labs– Focus on Detection/Tracking/Recognition

• GREEnMan

Why CV@NICTA is the best group to do this

• One of the strongest CV research groups in the world in terms of A+ and A publications at recent CV conferences (see appendices)

• One of the strongest CV research groups in the world in terms of citations (see appendices)

• One of the strongest groups in real-time structure from motion (see appendices)

• Dominant world group in Multi-body structure from motion

• Strong mix across research approaches in CV and pattern recognition

• World-class parallel real-time implementations (PMS)• World-class traffic surveillance systems• Outstanding linkages in safety and security

Contributors and acknowledgements

• Contributors– Nick Barnes, Abbas, Bigdeli, Terry Caelli, Richard Hartley,

Bernhard Hengst, Brian Lovell, Chris, Nicol, John Parker, Lars Petersson, Antonio Robles-Kelly, Jian Zhang

• Discussions with the following people greatly contributed to this plan:– Emma Barron, David Everitt, Terry Percival, Phil

Robertson, David Skellern, Chris Scott, Bob Williamson