robust situation awareness: from active sensing … · robust situation awareness: from active...
TRANSCRIPT
![Page 1: Robust Situation Awareness: From Active Sensing … · Robust Situation Awareness: From Active Sensing to Coherent Active Perception Dr David Johnson Acknowledgements: Robert Fitch,](https://reader031.vdocuments.site/reader031/viewer/2022031008/5b9048fa09d3f2857e8b9b6a/html5/thumbnails/1.jpg)
Robust Situation Awareness: From
Active Sensing to Coherent Active
Perception
Dr David Johnson
Acknowledgements: Robert Fitch, James
Underwood, Thierry Peynot, Trevor Anderson,
James Spedding and many others
![Page 2: Robust Situation Awareness: From Active Sensing … · Robust Situation Awareness: From Active Sensing to Coherent Active Perception Dr David Johnson Acknowledgements: Robert Fitch,](https://reader031.vdocuments.site/reader031/viewer/2022031008/5b9048fa09d3f2857e8b9b6a/html5/thumbnails/2.jpg)
![Page 3: Robust Situation Awareness: From Active Sensing … · Robust Situation Awareness: From Active Sensing to Coherent Active Perception Dr David Johnson Acknowledgements: Robert Fitch,](https://reader031.vdocuments.site/reader031/viewer/2022031008/5b9048fa09d3f2857e8b9b6a/html5/thumbnails/3.jpg)
Background: How does the ACFR fit in?
![Page 4: Robust Situation Awareness: From Active Sensing … · Robust Situation Awareness: From Active Sensing to Coherent Active Perception Dr David Johnson Acknowledgements: Robert Fitch,](https://reader031.vdocuments.site/reader031/viewer/2022031008/5b9048fa09d3f2857e8b9b6a/html5/thumbnails/4.jpg)
Overview
• Motivation
• Background
• Some Definitions
• Sensing Technology–Radar / Lidar
–Machine Vision
–Coherent Sensor Fusion
• Application Examples–Mining
–Agriculture / Asset Monitoring
–Security / Defense
• Future Directions
• Shameless Plug
![Page 5: Robust Situation Awareness: From Active Sensing … · Robust Situation Awareness: From Active Sensing to Coherent Active Perception Dr David Johnson Acknowledgements: Robert Fitch,](https://reader031.vdocuments.site/reader031/viewer/2022031008/5b9048fa09d3f2857e8b9b6a/html5/thumbnails/5.jpg)
Motivation: Field Robotics in
Australia
• Australia leads the world in civilian field-robotic applications– Dirty, Dull and Dangerous is a way-of-life
– Big country
– Small population Innovate or perish
– Low subsidies
• Mining: Rio Tinto’s ‘Mine of the Future’ program– World’s largest non-military robotics programme
• Agriculture: 1st IEEE Agricultural Robotics Summer School held in Sydney, Feb 2015
• Logistics: Port of Brisbane (and now Port Botany)– World’s 1st fully-automated container-handling terminal
David Johnson | Australian Centre for Field Robotics
![Page 6: Robust Situation Awareness: From Active Sensing … · Robust Situation Awareness: From Active Sensing to Coherent Active Perception Dr David Johnson Acknowledgements: Robert Fitch,](https://reader031.vdocuments.site/reader031/viewer/2022031008/5b9048fa09d3f2857e8b9b6a/html5/thumbnails/6.jpg)
Some Definitions
• Autonomy is the automation of independent perception,
learning, control and action.
• Humans are very good at controlling remote vehicles using
low-quality visual data, as seen by FPV-racing.
David Johnson | Australian Centre for Field Robotics 6
• Much of this is instinctual,
using context and experience
to ‘fill in the gaps’ and guess
what will happen next.
• It is not surprising that these
tasks are difficult to automate,
but we can make the sensors
better, the processing faster,
etc.
![Page 7: Robust Situation Awareness: From Active Sensing … · Robust Situation Awareness: From Active Sensing to Coherent Active Perception Dr David Johnson Acknowledgements: Robert Fitch,](https://reader031.vdocuments.site/reader031/viewer/2022031008/5b9048fa09d3f2857e8b9b6a/html5/thumbnails/7.jpg)
David Johnson | Australian Centre for Field Robotics 7
Perception Challenges
Scale (complexity)
Variability (novelty)
Interaction (not a black box)
![Page 8: Robust Situation Awareness: From Active Sensing … · Robust Situation Awareness: From Active Sensing to Coherent Active Perception Dr David Johnson Acknowledgements: Robert Fitch,](https://reader031.vdocuments.site/reader031/viewer/2022031008/5b9048fa09d3f2857e8b9b6a/html5/thumbnails/8.jpg)
Perception Challenge #1:
Scale
Space - Localisation at the mm scale across km-wide maps.
Time - Online processing with real-time requirements for high speed ground, air and water-based vehicles with revisit-times ranging from ms to years.
Frequency - Coherent sensing and communications from kHz to THz, optical and beyond
Number – Dimensionality of many different variables
David Johnson | Australian Centre for Field Robotics
![Page 9: Robust Situation Awareness: From Active Sensing … · Robust Situation Awareness: From Active Sensing to Coherent Active Perception Dr David Johnson Acknowledgements: Robert Fitch,](https://reader031.vdocuments.site/reader031/viewer/2022031008/5b9048fa09d3f2857e8b9b6a/html5/thumbnails/9.jpg)
Perception Challenge #2:
Variability
David Johnson | Australian Centre for Field Robotics
• Challenges for computer-vision:– Illumination– Object pose– Clutter– Occlusions
• Challenges for active sensors– Attenuation– Multipath– Interference– Sample-aliasing
![Page 10: Robust Situation Awareness: From Active Sensing … · Robust Situation Awareness: From Active Sensing to Coherent Active Perception Dr David Johnson Acknowledgements: Robert Fitch,](https://reader031.vdocuments.site/reader031/viewer/2022031008/5b9048fa09d3f2857e8b9b6a/html5/thumbnails/10.jpg)
10
Atmospheric attenuation
DRIZZLE (0.25mm/hr)
SUBMILLIMETERMILLIMETER
1003mm
1 THz0.3mm
1030m
1003m
10000.3m
VISIBLEINFRARED
10 GHz3cm
0.01
0.1
1
10
100
1000
H2O
O2
H2O
O2
HEAVY RAIN(25mm/hr)
DELUGE(150mm/hr)
FOG (0.1g/m3)
VISIBILITY (50m)
CO2
H2O
CO2
CO2
H2O
H2O
O3
AT
TE
NU
AT
ION
(dB
/km
)
20Atm
Preissner, J. (1978, The Influence of the Atmosphere on Passive
Radiometric Measurements. AGARD Conference Reprint No. 245:
Millimeter and Submillimeter Wave Propagation and Circuits.David Johnson | Australian Centre for Field Robotics
![Page 11: Robust Situation Awareness: From Active Sensing … · Robust Situation Awareness: From Active Sensing to Coherent Active Perception Dr David Johnson Acknowledgements: Robert Fitch,](https://reader031.vdocuments.site/reader031/viewer/2022031008/5b9048fa09d3f2857e8b9b6a/html5/thumbnails/11.jpg)
Perception Challenge #3:
Interaction
David Johnson | Australian Centre for Field Robotics
• Human Control Interface – User
interaction, levels of autonomy
• Human Environment – Co-habitation vs.
isolation
• Natural Environment – Manipulation of
and attenuation by
• Machine Interaction – Systems of
Systems, collaboration, communication
![Page 12: Robust Situation Awareness: From Active Sensing … · Robust Situation Awareness: From Active Sensing to Coherent Active Perception Dr David Johnson Acknowledgements: Robert Fitch,](https://reader031.vdocuments.site/reader031/viewer/2022031008/5b9048fa09d3f2857e8b9b6a/html5/thumbnails/12.jpg)
Some Definitions
• Robustness• Situational
Awareness
• Active Sensing
• Coherence
• Active Perception
David Johnson | Australian Centre for Field Robotics 12
• Robust to purpose (reliability)
• Minimal false-negatives / false-
positives
• Robust to the environment
(persistence)
• Minimal unknown failure modes
• Onboard diagnostics
• Robust to the (less-skilled)
operator (trust)
• Functional safety
• Intuitive operation
• Robust to scale (cost)
• Designed for mass-production
• Full product life-cycle
considerations
• Spectrum congestion
• Standards compliance
• ‘Big-Data’ processing
![Page 13: Robust Situation Awareness: From Active Sensing … · Robust Situation Awareness: From Active Sensing to Coherent Active Perception Dr David Johnson Acknowledgements: Robert Fitch,](https://reader031.vdocuments.site/reader031/viewer/2022031008/5b9048fa09d3f2857e8b9b6a/html5/thumbnails/13.jpg)
Some Definitions
• Robustness
• Situational Awareness
• Active Sensing
• Coherence
• Active Perception
David Johnson | Australian Centre for Field Robotics 13
![Page 14: Robust Situation Awareness: From Active Sensing … · Robust Situation Awareness: From Active Sensing to Coherent Active Perception Dr David Johnson Acknowledgements: Robert Fitch,](https://reader031.vdocuments.site/reader031/viewer/2022031008/5b9048fa09d3f2857e8b9b6a/html5/thumbnails/14.jpg)
Some Definitions
• Robustness
• Situational Awareness
• Active Sensing
• Coherence
• Active Perception
David Johnson | Australian Centre for Field Robotics 14
Active sensors put a controlled
signal into the environment, and
measure the interactions between
that signal and the environment to
infer information.
Examples:
• RADAR, LIDAR, X-ray Imaging
… use the EM field
• SONAR, Seismic
… use pressure waves
14
![Page 15: Robust Situation Awareness: From Active Sensing … · Robust Situation Awareness: From Active Sensing to Coherent Active Perception Dr David Johnson Acknowledgements: Robert Fitch,](https://reader031.vdocuments.site/reader031/viewer/2022031008/5b9048fa09d3f2857e8b9b6a/html5/thumbnails/15.jpg)
15
• A portion of the EM spectrum is transmitted, reflects from a
‘scatterer’, and gets picked up by a receiver.
• 2 ways of forming high-range-resolution:
– A very short pulse in time (TOF)
– A very wide swept-frequency-bandwidth (FMCW)
• Hence to achieve 10mm resolution requires either a pulse width of
<0.1ns, or a swept bandwidth of 15GHz
15
Range Resolution = Speed of Light
2 x Bandwidth
Range Resolution = Speed of Light x Pulse Width
2
Active Sensing Primer
David Johnson | Australian Centre for Field Robotics
![Page 16: Robust Situation Awareness: From Active Sensing … · Robust Situation Awareness: From Active Sensing to Coherent Active Perception Dr David Johnson Acknowledgements: Robert Fitch,](https://reader031.vdocuments.site/reader031/viewer/2022031008/5b9048fa09d3f2857e8b9b6a/html5/thumbnails/16.jpg)
16
(Linear) FM or ToF Waveforms
Fre
qu
en
cy
Time
Tchirp
sweepF
cR
.2
chirpTv
.2
Fstep
Fsweep
=N.Fstep
• Range resolution improves with
swept bandwidth
• Velocity resolution improves with
sweep time
• Range to a target is measured by
running a counter from the time a pulse
is transmitted until the echo is received.
• The echo is detected when the received
signal envelope exceeds a
predetermined threshold
• Knowing the signal propagation velocity
it is possible to calculate the range to
the target
David Johnson | Australian Centre for Field Robotics
![Page 17: Robust Situation Awareness: From Active Sensing … · Robust Situation Awareness: From Active Sensing to Coherent Active Perception Dr David Johnson Acknowledgements: Robert Fitch,](https://reader031.vdocuments.site/reader031/viewer/2022031008/5b9048fa09d3f2857e8b9b6a/html5/thumbnails/17.jpg)
17
Equivalent resolution using AM or FM
David Johnson | Australian Centre for Field Robotics
![Page 18: Robust Situation Awareness: From Active Sensing … · Robust Situation Awareness: From Active Sensing to Coherent Active Perception Dr David Johnson Acknowledgements: Robert Fitch,](https://reader031.vdocuments.site/reader031/viewer/2022031008/5b9048fa09d3f2857e8b9b6a/html5/thumbnails/18.jpg)
ACFR Radar Sensors
![Page 19: Robust Situation Awareness: From Active Sensing … · Robust Situation Awareness: From Active Sensing to Coherent Active Perception Dr David Johnson Acknowledgements: Robert Fitch,](https://reader031.vdocuments.site/reader031/viewer/2022031008/5b9048fa09d3f2857e8b9b6a/html5/thumbnails/19.jpg)
19
Examples of Lidar Sensors
HDL-64E HDL-32E VLP-16
# Lasers 64 32 16
Max Range (10% refl) 50m 50m 50m
Field of View (H / V) 360 / -24.8 to +2 360 / -30 to +10 360 / -15 to +15
Scan Rate 5 - 15 Hz 5 – 20 Hz 10 Hz
Points per Scan ~140k ~70k ~30k (x2)
~ Cost (AUD) ~$60k ~$30k ~$10k
Communications Standard Ethernet (UDP) Ethernet (UDP) Ethernet (UDP)
David Johnson | Australian Centre for Field Robotics
![Page 20: Robust Situation Awareness: From Active Sensing … · Robust Situation Awareness: From Active Sensing to Coherent Active Perception Dr David Johnson Acknowledgements: Robert Fitch,](https://reader031.vdocuments.site/reader031/viewer/2022031008/5b9048fa09d3f2857e8b9b6a/html5/thumbnails/20.jpg)
20
More Examples of Lidar Sensors
UTM-30LX VZ-1000 LMS-Z620
# Lasers 1 1 1
Max Range (10% refl) 30m (cal board) 700m 750m
Field of View (H / V) 270 / N.A. 360 / 100 360 / 80
Scan Rate 40 Hz 120 V-lines / sec 20 V-lines / sec
Points per Scan ~1k 122000 pts / sec 11000 pts / sec
~ Cost (AUD) ~$5k ~$250k ~$150k
Communications Standard USB 2.0 GbE 100Mbps Ethernet
David Johnson | Australian Centre for Field Robotics
![Page 21: Robust Situation Awareness: From Active Sensing … · Robust Situation Awareness: From Active Sensing to Coherent Active Perception Dr David Johnson Acknowledgements: Robert Fitch,](https://reader031.vdocuments.site/reader031/viewer/2022031008/5b9048fa09d3f2857e8b9b6a/html5/thumbnails/21.jpg)
Some Definitions
• Robustness
• Situational Awareness
• Active Sensing
• Coherence
• Active Perception
21
Bajcsy (1998) - Active Perception
“Intelligent data acquisition”
[Closing the perception-action loop]
Bottom-up: Data driven
Top-down: Goal driven
Haykin (2006) - Cognitive Radar Networks
David Johnson | Australian Centre for Field Robotics
![Page 22: Robust Situation Awareness: From Active Sensing … · Robust Situation Awareness: From Active Sensing to Coherent Active Perception Dr David Johnson Acknowledgements: Robert Fitch,](https://reader031.vdocuments.site/reader031/viewer/2022031008/5b9048fa09d3f2857e8b9b6a/html5/thumbnails/22.jpg)
Additional Information in the data
David Johnson | Australian Centre for Field Robotics 22
![Page 23: Robust Situation Awareness: From Active Sensing … · Robust Situation Awareness: From Active Sensing to Coherent Active Perception Dr David Johnson Acknowledgements: Robert Fitch,](https://reader031.vdocuments.site/reader031/viewer/2022031008/5b9048fa09d3f2857e8b9b6a/html5/thumbnails/23.jpg)
Some Definitions
• Robustness
• Situational Awareness
• Active Sensing
• Coherence• Active
Perception
David Johnson | Australian Centre for Field Robotics
• Coherent (multi-modal) sensing– Correlation of pre-processed data to
associate data across multiple spectral bands.
• Coherent Active Perception used to adapt sensor parameters to the environment. Based on:
• Context
• Experience (prior information)
• External cues
• For example: Vision-guided RF tomography– UWB radar prone to boundary-layer
disturbance
– Structured-light or Stereo Vision provides highly accurate geometric surface model
– Combined solution allows this disturbance to be removed from the raw radar signal
![Page 24: Robust Situation Awareness: From Active Sensing … · Robust Situation Awareness: From Active Sensing to Coherent Active Perception Dr David Johnson Acknowledgements: Robert Fitch,](https://reader031.vdocuments.site/reader031/viewer/2022031008/5b9048fa09d3f2857e8b9b6a/html5/thumbnails/24.jpg)
Multispectral
Hyper-spectral
Dr Sildomar Monteiro 24Cafe Seminar RTCMA
Band 4Band 5
Band 6
Band 2Band 3
Band n
Band 1
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
500 750 1000 1250 1500 1750 2000 2250 2500
Reflec
tance
Wavelength
samples_644-17_1_00000.asd.ref samples_644-17_1_00035.asd.ref
Multi-Modal Sensor Fusion:
Lidar & Hyper-Spectral
![Page 25: Robust Situation Awareness: From Active Sensing … · Robust Situation Awareness: From Active Sensing to Coherent Active Perception Dr David Johnson Acknowledgements: Robert Fitch,](https://reader031.vdocuments.site/reader031/viewer/2022031008/5b9048fa09d3f2857e8b9b6a/html5/thumbnails/25.jpg)
Multi Modal Sensor Fusion:
Lidar & Radar
James Underwood | Australian Centre for Field Robotics
• Multi-modal resilient perception
• Simultaneous data association and classification
![Page 26: Robust Situation Awareness: From Active Sensing … · Robust Situation Awareness: From Active Sensing to Coherent Active Perception Dr David Johnson Acknowledgements: Robert Fitch,](https://reader031.vdocuments.site/reader031/viewer/2022031008/5b9048fa09d3f2857e8b9b6a/html5/thumbnails/26.jpg)
Optically Derived Simulation
Environment
• Complex target signatures remain poorly understood– Polarisation
– Prevalence of Diffraction
– Electrically large scenes
• MMW Radar and Side-scan Sonar operate in the optical regime within their respective domains
• Optical ray-tracing renderers therefore provide the capability for developing radar/sonar simulation engines to assist:– Characterisation of existing sensors
– Understanding of complex scatterer behaviour
– A means to potentially identify and classify targets
26Trevor Anderson, David Johnson | Australian Centre for Field Robotics
![Page 27: Robust Situation Awareness: From Active Sensing … · Robust Situation Awareness: From Active Sensing to Coherent Active Perception Dr David Johnson Acknowledgements: Robert Fitch,](https://reader031.vdocuments.site/reader031/viewer/2022031008/5b9048fa09d3f2857e8b9b6a/html5/thumbnails/27.jpg)
Our Approach
• Within an existing ray-tracing renderer ‘pbrt’ we
implemented both physical and geometric optics
modifications to provide:
– Frequency Dependence
– Polarisation Dependence
– Phase Dependence and Interference
– Geometric Edge Diffraction
– Of these, diffraction is the main stumbling block, requiring
quite sophisticated edge detection.
– The Uniform Theory of Diffraction [Kouyoumjian and Pathak]
is then applied, however finding a smooth transition between
specular and diffractive behaviour proved elusive.
27Trevor Anderson, David Johnson | Australian Centre for Field Robotics
![Page 28: Robust Situation Awareness: From Active Sensing … · Robust Situation Awareness: From Active Sensing to Coherent Active Perception Dr David Johnson Acknowledgements: Robert Fitch,](https://reader031.vdocuments.site/reader031/viewer/2022031008/5b9048fa09d3f2857e8b9b6a/html5/thumbnails/28.jpg)
Experimental Validation
28
• 94GHz FMCW radar (on its side)– 5 deg elevation (now azimuth) beam-width
– 0.5 deg azimuth (now elevation) beam-width
– 7cm range-resolution
• 90mm (Power-) Cube
• 150mm dihedral reflector
• 1:40 Scale model haul-truck
Trevor Anderson, David Johnson | Australian Centre for Field Robotics
![Page 29: Robust Situation Awareness: From Active Sensing … · Robust Situation Awareness: From Active Sensing to Coherent Active Perception Dr David Johnson Acknowledgements: Robert Fitch,](https://reader031.vdocuments.site/reader031/viewer/2022031008/5b9048fa09d3f2857e8b9b6a/html5/thumbnails/29.jpg)
Rotating Cube
• PowerCube Actuator, rotational accuracy of 0.01 deg
29
![Page 30: Robust Situation Awareness: From Active Sensing … · Robust Situation Awareness: From Active Sensing to Coherent Active Perception Dr David Johnson Acknowledgements: Robert Fitch,](https://reader031.vdocuments.site/reader031/viewer/2022031008/5b9048fa09d3f2857e8b9b6a/html5/thumbnails/30.jpg)
Scale Model Haul Truck
30Trevor Anderson, David Johnson | Australian Centre for Field Robotics
![Page 31: Robust Situation Awareness: From Active Sensing … · Robust Situation Awareness: From Active Sensing to Coherent Active Perception Dr David Johnson Acknowledgements: Robert Fitch,](https://reader031.vdocuments.site/reader031/viewer/2022031008/5b9048fa09d3f2857e8b9b6a/html5/thumbnails/31.jpg)
From Radar to Sonar
• DST Group DSP project: Real-time Sonar Simulation
• Based on Nvidia Optix GPU
Trevor Anderson, David Johnson | Australian Centre for Field Robotics
![Page 32: Robust Situation Awareness: From Active Sensing … · Robust Situation Awareness: From Active Sensing to Coherent Active Perception Dr David Johnson Acknowledgements: Robert Fitch,](https://reader031.vdocuments.site/reader031/viewer/2022031008/5b9048fa09d3f2857e8b9b6a/html5/thumbnails/32.jpg)
Ground-Truthed Virtual Environment
32
Beall, Chris, et al. "Bundle adjustment in large-scale 3D
reconstructions based on underwater robotic surveys." OCEANS,
2011 IEEE-Spain. IEEE, 2011.
Trevor Anderson, David Johnson | Australian Centre for Field Robotics
![Page 33: Robust Situation Awareness: From Active Sensing … · Robust Situation Awareness: From Active Sensing to Coherent Active Perception Dr David Johnson Acknowledgements: Robert Fitch,](https://reader031.vdocuments.site/reader031/viewer/2022031008/5b9048fa09d3f2857e8b9b6a/html5/thumbnails/33.jpg)
Complex Object Simulation
• Vertices: 206113
• Faces: 389552
• File size: 36.5 MB
33Trevor Anderson, David Johnson | Australian Centre for Field Robotics
![Page 34: Robust Situation Awareness: From Active Sensing … · Robust Situation Awareness: From Active Sensing to Coherent Active Perception Dr David Johnson Acknowledgements: Robert Fitch,](https://reader031.vdocuments.site/reader031/viewer/2022031008/5b9048fa09d3f2857e8b9b6a/html5/thumbnails/34.jpg)
Integration with MOOS
34
Simulated Sensors
Vehicle Position
Estimator
Side Scan Simulator
Vehicle Dynamics
(Simulated Position)
Detection Algorithm
Vehicle Visualisation
Tool
Sonar Visualiser
Autonomy Unit
Trevor Anderson, David Johnson | Australian Centre for Field Robotics
![Page 35: Robust Situation Awareness: From Active Sensing … · Robust Situation Awareness: From Active Sensing to Coherent Active Perception Dr David Johnson Acknowledgements: Robert Fitch,](https://reader031.vdocuments.site/reader031/viewer/2022031008/5b9048fa09d3f2857e8b9b6a/html5/thumbnails/35.jpg)
Integration with MOOS
35Trevor Anderson, David Johnson | Australian Centre for Field Robotics
![Page 36: Robust Situation Awareness: From Active Sensing … · Robust Situation Awareness: From Active Sensing to Coherent Active Perception Dr David Johnson Acknowledgements: Robert Fitch,](https://reader031.vdocuments.site/reader031/viewer/2022031008/5b9048fa09d3f2857e8b9b6a/html5/thumbnails/36.jpg)
Result - Video
36Trevor Anderson, David Johnson | Australian Centre for Field Robotics
![Page 37: Robust Situation Awareness: From Active Sensing … · Robust Situation Awareness: From Active Sensing to Coherent Active Perception Dr David Johnson Acknowledgements: Robert Fitch,](https://reader031.vdocuments.site/reader031/viewer/2022031008/5b9048fa09d3f2857e8b9b6a/html5/thumbnails/37.jpg)
Using a Ray Traced simulator to perform
model based classification.
37
Not mine!
Mine, Rockan!
Area Surveying – Data Gathering MLO Detection Classification
• 2nd DSP Project: Model-based closed-loop sonar perception
Trevor Anderson, David Johnson | Australian Centre for Field Robotics
![Page 38: Robust Situation Awareness: From Active Sensing … · Robust Situation Awareness: From Active Sensing to Coherent Active Perception Dr David Johnson Acknowledgements: Robert Fitch,](https://reader031.vdocuments.site/reader031/viewer/2022031008/5b9048fa09d3f2857e8b9b6a/html5/thumbnails/38.jpg)
Mugshot Simulator
• Only interested in mugshots Only simulate
mugshots
• Effectively assumes detection is a solved problem
– Most likely as a result of 1st pass detection by existing ATR
algorithm
• Research Question: Can we determine a path that
best discriminates between one type and another?
38
Cube Manta ’Cylinder’ Rockan
Trevor Anderson, David Johnson | Australian Centre for Field Robotics
![Page 39: Robust Situation Awareness: From Active Sensing … · Robust Situation Awareness: From Active Sensing to Coherent Active Perception Dr David Johnson Acknowledgements: Robert Fitch,](https://reader031.vdocuments.site/reader031/viewer/2022031008/5b9048fa09d3f2857e8b9b6a/html5/thumbnails/39.jpg)
Interactive Mugshot Visualisation
• Often the effect of a parameter on the sonar image
isn’t intuitive
• Human intuition is developed around interacting with
things – making a change and seeing the response
39Trevor Anderson, David Johnson | Australian Centre for Field Robotics
![Page 40: Robust Situation Awareness: From Active Sensing … · Robust Situation Awareness: From Active Sensing to Coherent Active Perception Dr David Johnson Acknowledgements: Robert Fitch,](https://reader031.vdocuments.site/reader031/viewer/2022031008/5b9048fa09d3f2857e8b9b6a/html5/thumbnails/40.jpg)
• Require parameter independent classification…
• Sonar imaging process is not determinate
– Require probabilistic method
• Has many parallels to image identification
40
The Classification Problem
Cylinder:
Rockan:
??
Trevor Anderson, David Johnson | Australian Centre for Field Robotics
![Page 41: Robust Situation Awareness: From Active Sensing … · Robust Situation Awareness: From Active Sensing to Coherent Active Perception Dr David Johnson Acknowledgements: Robert Fitch,](https://reader031.vdocuments.site/reader031/viewer/2022031008/5b9048fa09d3f2857e8b9b6a/html5/thumbnails/41.jpg)
Image Comparison
• 2D Cross Correlation
41Trevor Anderson, David Johnson | Australian Centre for Field Robotics
![Page 42: Robust Situation Awareness: From Active Sensing … · Robust Situation Awareness: From Active Sensing to Coherent Active Perception Dr David Johnson Acknowledgements: Robert Fitch,](https://reader031.vdocuments.site/reader031/viewer/2022031008/5b9048fa09d3f2857e8b9b6a/html5/thumbnails/42.jpg)
Gibbs Sampling the full parameter
space.
42
• Randomly samples
the parameter space
• Samples are rejected
according to the
correlation value
• Final sample
distribution reflects
probability density
Trevor Anderson, David Johnson | Australian Centre for Field Robotics
![Page 43: Robust Situation Awareness: From Active Sensing … · Robust Situation Awareness: From Active Sensing to Coherent Active Perception Dr David Johnson Acknowledgements: Robert Fitch,](https://reader031.vdocuments.site/reader031/viewer/2022031008/5b9048fa09d3f2857e8b9b6a/html5/thumbnails/43.jpg)
Non-Destructive Evaluation of Power
Poles using RF Tomography
David Johnson | Australian Centre for Field Robotics 43
This technology is portable, non-ionising, potentially license-free and capable of
high-resolution penetrative imaging over short distances
5 million wooden power poles in
Australia
~1 billion worldwide
![Page 44: Robust Situation Awareness: From Active Sensing … · Robust Situation Awareness: From Active Sensing to Coherent Active Perception Dr David Johnson Acknowledgements: Robert Fitch,](https://reader031.vdocuments.site/reader031/viewer/2022031008/5b9048fa09d3f2857e8b9b6a/html5/thumbnails/44.jpg)
Transmissive results
David Johnson | Australian Centre for Field Robotics 44
![Page 45: Robust Situation Awareness: From Active Sensing … · Robust Situation Awareness: From Active Sensing to Coherent Active Perception Dr David Johnson Acknowledgements: Robert Fitch,](https://reader031.vdocuments.site/reader031/viewer/2022031008/5b9048fa09d3f2857e8b9b6a/html5/thumbnails/45.jpg)
More Ground Truth Required: Faculty of
Health Sciences owed us a favour
45David Johnson | Australian Centre for Field Robotics
![Page 46: Robust Situation Awareness: From Active Sensing … · Robust Situation Awareness: From Active Sensing to Coherent Active Perception Dr David Johnson Acknowledgements: Robert Fitch,](https://reader031.vdocuments.site/reader031/viewer/2022031008/5b9048fa09d3f2857e8b9b6a/html5/thumbnails/46.jpg)
Reflection-mode scanning
![Page 47: Robust Situation Awareness: From Active Sensing … · Robust Situation Awareness: From Active Sensing to Coherent Active Perception Dr David Johnson Acknowledgements: Robert Fitch,](https://reader031.vdocuments.site/reader031/viewer/2022031008/5b9048fa09d3f2857e8b9b6a/html5/thumbnails/47.jpg)
Next generation system: Multi-modal
• Rotate around pole rather than vice versa.
• Arm kinematics don’t allow in-plane rotation –hence require actuated carriages.
• Each carriage equipped with stereo-vision sensor to reconstruct external geometry and detect deformations.
David Johnson | Australian Centre for Field Robotics 47
• Bistatic reflection-mode images use geometry to
allow phase/amplitude correction of waveform – true
coherent active perception system!
![Page 48: Robust Situation Awareness: From Active Sensing … · Robust Situation Awareness: From Active Sensing to Coherent Active Perception Dr David Johnson Acknowledgements: Robert Fitch,](https://reader031.vdocuments.site/reader031/viewer/2022031008/5b9048fa09d3f2857e8b9b6a/html5/thumbnails/48.jpg)
Perimeter Surveillance
• Detection of low RCS targets (<-40dBm2) at 2 - 50m,
in clutter.
• Low-cost Doppler-radar mesh network
– Currently 2, building 20 more
David Johnson | Australian Centre for Field Robotics 48
![Page 49: Robust Situation Awareness: From Active Sensing … · Robust Situation Awareness: From Active Sensing to Coherent Active Perception Dr David Johnson Acknowledgements: Robert Fitch,](https://reader031.vdocuments.site/reader031/viewer/2022031008/5b9048fa09d3f2857e8b9b6a/html5/thumbnails/49.jpg)
Summary
• Perception is not a solved problem – particularly
when robust operation is essential.
• Active sensing is often complementary to passive
(vision-based sensing), but more effort
• Active/Cognitive/Agile Sensing/Perception/Radar is a
mean to close the action-perception loop
– Adapting the system to seek refuting evidence of a
hypothesis
• Various examples of how these ideas are being
actively researched in the defence, mining, asset-
monitoring and security domains have been shown.
– I’m interested in solving your real-world problems (with radar)
David Johnson | Australian Centre for Field Robotics 49
![Page 50: Robust Situation Awareness: From Active Sensing … · Robust Situation Awareness: From Active Sensing to Coherent Active Perception Dr David Johnson Acknowledgements: Robert Fitch,](https://reader031.vdocuments.site/reader031/viewer/2022031008/5b9048fa09d3f2857e8b9b6a/html5/thumbnails/50.jpg)
Future Directions
• System-level rather than platform-level sensor coherence.
• Dealing with those robustness issues identified at the start, particularly decentralised processing and communications
• Dealing with levels of autonomy – balancing coupled action/perception/learning with high-level tasks
• Dealing with complex dynamic environments
• Dealing with adversity• (Didn’t mention my student projects on semi-autonomous FPV
racing!)
• LIVE-ACTION (WAR) ROBOT GAMES.
David Johnson | Australian Centre for Field Robotics 50
![Page 51: Robust Situation Awareness: From Active Sensing … · Robust Situation Awareness: From Active Sensing to Coherent Active Perception Dr David Johnson Acknowledgements: Robert Fitch,](https://reader031.vdocuments.site/reader031/viewer/2022031008/5b9048fa09d3f2857e8b9b6a/html5/thumbnails/51.jpg)
IROS 2016 Workshop:
State Estimation and Terrain Perception
Topics of interest (including…)
• Onboard sensing for 3D mapping of unstructured environments
• Modelling and handling of uncertainties, drift, and outliers
• Terrain representation and estimation
• Integration of dynamical models and kinematic constraints
• Integration of prior knowledge (terrain type, maps, beacons)
• Dealing with dynamic environments
• Sensor failures, reliability and redundancy
• Tackling difficulties in outdoor perception (sunlight, low light, smoke, rain)
• Experimental results and full system integration in real world applications
David Johnson | Australian Centre for Field Robotics51
![Page 52: Robust Situation Awareness: From Active Sensing … · Robust Situation Awareness: From Active Sensing to Coherent Active Perception Dr David Johnson Acknowledgements: Robert Fitch,](https://reader031.vdocuments.site/reader031/viewer/2022031008/5b9048fa09d3f2857e8b9b6a/html5/thumbnails/52.jpg)
David Johnson | Australian Centre for Field Robotics 52
Questions?