practical aspects of uas-based point cloud generation · practical aspects of uas-based point cloud...
TRANSCRIPT
Practical Aspects of UAS-based Point Cloud Generation
OTEC 2015 ConferenceOctober 27–30, 2015, Columbus, Ohio
Greg Jozkow
Acknowledgement to Charles Toth
2
Introduction
Geospatial data acquisition (GIS) with small UAS (sUAS)
AdvantagesInexpensive platformLightweight vehicle and sensorsEasy deployment and operationMinimal logisticsRelatively high resolution data
ChallengesLimited payloadLimited operating rangeShort flight timeModest/low sensor performanceAccurate georeferencing (e.g. direct georeferencing needed for LiDAR)
3
Typical Workflow for UAS‐based GIS Data
Present practice: Data acquisition includes
Mainly optical image capture Limited logging of georeferencing data
Sensor orientation is based on image block adjustment; primarily using ground control and rarely air control
Dense image matching is used to generate point cloud Point cloud processing includes filtering, DEM generation, segmentation Orthoimage and mosaic generation Quality control, accuracy assessment
Ongoing developments: Improving sensor technologies; e.g., UAS‐specific LiDAR or HSI camera More georeferencing options; e.g., dual‐frequency GPS, tactical grade IMU
Research topics and directions: Impact of GPS/IMU on point cloud generation Photogrammetrically derived point cloud vs. LiDAR point cloud
4
UAS Point Clouds
Laser scanning
Dense image matching
5
UAS Point Clouds
Semi‐Global Matching (SGM), pixel‐based matching, based on the disparity optimization
First term: matching cost pixelwise Second term: penalty for all pixels with different disparity (D)
Matching cost was MI, now is Census (Hamming distance) Fast implementation (GPU, FPGA)
H. Hirschmuller, Semi‐Global Matching – Motivation, Developments and Applications, PhoWo 2011, Ed. D. Fritsch
6
(b) Indirect georeferencing
Imaging is used for orientation first (past practice) and SLAM (presently) and then for mapping, reconnaissance, etc.
Georeferencing/Navigation Alternatives
(a) Direct georeferencing
Imaging is used formapping, reconnaissance, etc.
(c) Integrated sensor orientation (ISO)
Imaging is used for TRN and orientation first (SLAM), and then for mapping, reconnaissance, etc.
UAS practice
7
Direct vs. Indirect Georeferencing
Direct georeferencing (high‐accuracy) Carrier‐phase differential GPS (cm‐level accuracy) Medium‐grade (tactical) IMU (3‐15°/h gyro) Optional sensors (magnetometer, barometer, etc.) Time synchronization of sensors is needed Sensor and intra‐sensor calibration, such as boresighting, are important
Indirect georeferencing Image overlap requirements Image feature extraction and matching Bundle block adjustment (AT, aerial triangulation) Ground control is needed (GPS‐surveyed natural and/or signalized targets) Camera self‐calibration capability
ISO georeferencing Combined benefits of both methods, platform navigation data can be used as air
control; provides most robust solution
8
Error Characterization
Direct orientation
Error propagation has an extrapolation character
IO errors are uncompensated (!)
Larger variances of reconstructed points
Indirect orientation (integrated sensor orientation)
Error propagation has an interpolation character
Interior orientation errors are absorbed by AT
Platform orientation accuracy is not of interest
Wrong IO and AT => optimal EO reconstruction
9
Error Characterization
10
Test Configuration: Navigation Sensors
Epson IMUM‐G362PDC1
Wookong‐M autopilot:L1 GPS, MEMS IMU,magnetometer(platform navigation)
Antcom L1/L2 antennaNovAtel OEM615 GPS
Garmin GPS‐18LVL1 GPS(Velodyne configuration)
Gigabyte GB‐BXi7‐4500PC (data recording)
Solmeta Geotagger N3L1 GPS(Nikon configuration)
MicroStrain IMU3DMGX3‐35 with:L1 GPS, magnetometer
11
Small Navigation UAS Sensors: GPS
Dual‐frequency GPSNovAtelOEM615
Event input in YesPPS out YesSingle Point L1 RMS 1.5 mSingle Point L1/L2 RMS 1.2 mTime accuracy 20 nsPower requirements 1 WWeight without /with equipment
24 gram300 gram
Single‐frequency GPS(autopilot systems)
u‐bloxLEA‐6H
No raw dataNo timingHorizontal position accuracy (without aiding)
2.5 m
Time pulse accuracy 30 nsPower requirements 121 mWWeight without /with equipment
2 gram17 gram
12
Small Navigation UAS Sensors: IMU
IMUEpsonM‐G362PDC1
MicroStrain3DM‐GX3‐35
Autopilotsensors
Analog DevicesADIS16364
Gyro bias 3 °/ h 18 °/ h > 15 °/ h 25 °/ hGyro random walk N/A 0.1 °/ h1/2 N/A 2 °/ h1/2
Accelerometer bias 40 mg < 100 mg > 60 mg 8 mg
Accelerometer noise 40 mg / Hz1/2100 mg / Hz1/2
> 250 g / Hz1/2
270 g / Hz1/2
Power requirements30 mAvia USB
200 mAvia USB
> 4 mA(IMU only)
49 mA
Weight without /with equipment
7 gram30 gram
23 gram200 gram > 50 gram
16 gram
13
Test Configuration: Imaging Sensors
Nikon D800 Full frame 36 Mpix DSLR camera Nikon Nikkor AF‐S 50 mm f/1.4G lens 1 s triggering (1 FPS) Fixed to the platform in nadir facing direction
GoPro Hero 3+ Black Edition 12 Mpix Non‐stock 5.4 mm lens 0.5 s triggering (2 FPS) Gyro stabilized gimbal (vertical images)
Velodyne HDL‐32E Usable returns up to 70 m 32 laser diodes 700,000 pints/s FOV: + 30°(front) to ‐ 10°(rear) @ 360°
14
Test Flights: Topographical Mapping
Site 1 2 3 4Flying height [m] 120 125 125 25Flying speed [m/s] 4 4 4 4No. of flight lines 2 2 2 3GSD [mm] Nikon/GoPro
12‐
12‐
12‐
2.57
Velodyne flight ‐ ‐ ‐ +No of GCPs 31 0 15 19
1 2 3
4
15
Results: GPS Receivers’ Performance at Site 4
Positioning accuracy [m]
Geotagger MicroStrain NovAtel Indirect georeferencing (Nikon images)
Positioning method
Single Point (code)
Single Point (code)
Differential (carrier‐phase)
Image bundle block adjustment based on 15
accurate GCPs
Accuracy According specification
Computed by receiver
Computed in RTKlib
Computed from residuals at GCPs
Horizontal 3.0 4.6 0.029 0.016
Vertical N/A 5.6 0.046 0.018
3D N/A 7.2 0.054 0.025
Indirect georeferencing solution (AT) was used subsequently as reference(note image center is not constrained in AT)
16
Results: Direct Georeferencing at Site 4
Position RMSE [m]
Horizon
tal
Vertical
3D
A 5.1 10.2 11.4
B 8.6 16.5 18.6
C 0.025 0.035 0.043
A – GeotaggerB – MicroStrainC – NovAtelReference (AT)
17
Results: Point Cloud Generation from Images at Site 4
Nikon 108 images (more Mpix) lower FOV (smaller area, more gaps) ~ 2.5M points (higher density) lower noise
GoPro 131 images (less Mpix) larger FOV (larger area, less gaps) ~ 0.6M points (lower density) high noise
3D RMSE0.07 m
3D RMSE0.77 m
18
Results: Nikon vs. GoPro, Site 4
19
Results: GoPro Image Limitations at Site 4
Strange image distortion caused by rolling shutter impossible to model additional blurResulting in poor AT, (3D RMSE: 0.42 m) noisy and poor point cloud
(3D RMSE: 0.77m)
20
Results: Point Cloud Generation by Dense Matching at Site 1
Software package A B CAverage point spacing [m] 0.03 0.01 0.03Average density [points/m2] 846 9999 5933D RMSE [m] 0.09 0.13 0.52
A B C
21
Results: Point Cloud Generation from LiDAR Data at Site 4
Ongoing work, georeferencing and calibration issues are not yet finished
10 m
0
298.0 m
295.0 m
292.5 m
290.0 m
287.5 m
285.0 m
283.0 m
Intensity
Elevation
Point cloud density [points/m2]Nikon 12,429GoPro 7,175Velodyne 867
160
128
96
64
32
22
Cross Comparison and Status
GCP GPS GPS/MEMS IMU Carrier‐based GPS/MEMS IMU Density
GoProYes Good1 Good1 Good1 High
No Fair2 Fair2 Good1 High
NikonYes Excellent Excellent Excellent High
No Fair3 Fair3 Excellent High
Velodyne N/A No No Good/Excellent (not yet validated) Medium
Point cloud generation performance w.r.t. georeferencingaccuracy and image sensor quality
1) Point cloud accuracy limited by imaging sensor performance2) Poor absolute accuracy and fair relative accuracy3) Poor absolute accuracy and good relative accuracy
23
Very high resolution Accurate georeferencing Features on wires Dense matching – point cloud
(similarity to LiDAR)
UAS imagesGSD 5 mm
(on wires)10 pix across the wire
Flying height55 m over wires
UAS Point Cloud Generation in non Topographic Applications
24
Data acquisition Image block adjustment
Image dense matching
(point cloud)
Point cloud filtering
Point cloud segmentation / wires detection
3D geometry extraction
(catenary fitting)
Hi‐res imagesHigh overlapGCPs and/or high quality navigation data
ATBundle adjustment
Variety of methods, e.g. SGM
Unwanted points removal
Groups of points belonging to single wire and sag
Horizontally – lineVertically – catenaryRobust LS
Semi‐manual in this investigation
Few software packages tested
3D Geometry Extraction: Workflow
25
Single section of transmission line between two poles Two levels of wires 2 wires at upper level U 3 wires at lower level L
Expected different azimuth for U2 wire due to different mount
Test Object
26
Camera Nikon D800 (full‐frame
36Mpix DSLR) Nikon Nikkor AF‐S 50
mm f/1.4G 1 FPS
10 GCPs GPS‐RTK 3D RMSE 1.5 cm
10 GCPs GPS‐RTK 3D RMSE 1.5 cm
Image param. Ground Wire
GSD [mm] 8 5‐6
Endlap [%] 87 81‐84
Sidelap [%] 69 55‐61
Test Data Acquisition
27
Number of pointsL1 L2 L3 U1 U2
46,021 25,423 35,247 17,415 3,259
Point Cloud Filtering and Segmentation
28
Results: Fitted Catenaries
Catenaries fitting 3D RMSE = 10 cm
29
Conclusions
Point clouds can be easily generated from UAS imagery; they are less/not dependent on the accuracy of platform direct georeferencing
Point cloud quality and accuracy are strongly dependent on the camera geometrical properties and image scale (cm‐ and dm‐level accuracy) as well as the matching method/implementation used
UAS point clouds have potential in non topographic applications
Typically used single‐frequency GPS receivers (single solution) are not suitable for UAS georeferencing for precise mapping purposes (m level accuracy)
Miniaturized dual‐frequency GPS receivers (differential solution) are fit for direct georeferencing of UAS (cm level accuracy)
Using ground control and/or accurate air control provides the highest orientation accuracy for all sensors
More investigation on MEMS IMU is needed on UAS (advancing technology) More work on high accuracy GPS/MEMS IMU solution is required to assess
LiDAR point cloud performance and compare it to image derived point clouds
30
THANK YOU!