point clouds from calibration to classification · calibration of point clouds. integrated...
TRANSCRIPT
Point CloudsFrom Calibration to Classification
Norbert Pfeifer
Technische Universität Wien, Department for Geodesy and GeoinformationVienna, Austria
ContributionsGottfried Mandlburger (IfP Stuttgart, Germany), Philipp Glira (Siemens, Austria), Martin Pfennigbauer (Riegl, Austria), Konrad Wenzel (nframes, Germany), Tran Giang
Point Cloud Calibration and Orientation
2
Dense Image Matching (RGB)Airborne LiDAR
• 3D color vs. mainly geometry• Differences: vegetation, crane• Piles of construction material• DIM: stereo occlucion • LiDAR: points between piles available
DIMLiDAR
How can these point cloudsbe „brought together“in order to understand their differences?
Mandlburger et al.
Point Cloud Change Detection
4 data sets taken at 4 different times Different modality
RGB, nIR, photo/lidar, GSD, point density, shadow, …
Classified Lidar point cloudsground, building, vegetation
32005 2011
Computational Geometry – Point Cloud Legacy
Design Process• Given: Point cloud pi ∈R3, i = 1, … n• Find: surface s(u,v) closely interpolating pi , i.e. minimize f ( s(ui,vi) - pi )• NURBS (…) surface, patch layout based on curvature, curvature defined in each point
5Image: learning alias
Computational Geometry – Point Cloud Legacy
Design Process• Given: Point cloud pi ∈R3, i = 1, … n• Find: surface s(u,v) closely interpolating pi , i.e. minimize f ( s(ui,vi) - pi )• NURBS (…) surface, patch layout based on curvature, curvature defined in each point
Quality Inspection• Given: CAD model of a object, discretized as points mi
• Given: point cloud of the manufactured object, data points dj
• Find: transformation of dj onto mi , but no exact correspondence• Replace exact with approximate, iteratively determined correspondence: ICP
6Image: learning alias
Computational Geometry – Point Cloud Legacy
Design Process• Given: Point cloud pi ∈R3, i = 1, … n• Find: surface s(u,v) closely interpolating pi , i.e. minimize f ( s(ui,vi) - pi )• NURBS (…) surface, patch layout based on curvature, curvature defined in each point
Quality Inspection• Given: CAD model of a object, discretized as points mi
• Given: point cloud of the manufactured object, data points dj
• Find: transformation of dj onto mi , but no exact correspondence• Replace exact with approximate, iteratively determined correspondence: ICP
Visualization• Given: Point cloud pi
• Find TIN with vertices pi that follows surface of object• No 2D solution, e.g. Delaunay criterion,
but localized processing independent of overall shape
7
Point clouds
Computational Geometry Object centered (often) Point pi ( xi , yi , zi ) Cartesian coordinate system No datum No attributes Error free (negligible errors) No measurement process
8
Geodesy and Geoinformation Billions of points Projected CS Random (0.5-30cm), systematic (up
to meter), and many gross errors Attributes: color, accuracy, echo ID,
FWF information, time, etc. Measurement process known Measurement along optical line of
sight
Point cloud research domain
Data structures kD-tree
appropriate for processing, nearestneighbor search, etc.
octreeappropriate for visualization (e.g. potree)
Data structure for manuel editing, processing and visualization of pointclouds?
9
Geodesy and Geoinformation Billions of points Projected CS Random (0.5-30cm), systematic (up
to meter), and many gross errors Attributes: color, accuracy, echo ID,
FWF information, time, etc. Measurement process known Measurement along optical line of
sight
Point cloud research domain
Data structures kD-tree
appropriate for processing, nearestneighbor search, etc.
octreeappropriate for visualization (e.g. potree)
Data structure for manuel editing, processing and visualization of pointclouds?
Processing strategiesparallelizationtile wise processing
Neighborhoodoptimal neighborhoods, parameterselection, etc.
10
Geodesy and Geoinformation Billions of points Projected CS Random (0.5-30cm), systematic (up
to meter), and many gross errors Attributes: color, accuracy, echo ID,
FWF information, time, etc. Measurement process known Measurement along optical line of
sight
Point cloud research domain
Calibration and orientation sensor modeling measurement process modeling
sensor + medium/media + object observation error models
12
Geodesy and Geoinformation Billions of points Projected CS Random (0.5-30cm), systematic (up
to meter), and many gross errors Attributes: color, accuracy, echo ID,
FWF information, time, etc. Measurement process known Measurement along optical line of
sight
Glira et al.Image: Riegl
Point cloud research domain
Applications land cover mapping topographic modeling indoor modeling engineering surveying deformation analysis change detection
13
Geodesy and Geoinformation Billions of points Projected CS Random (0.5-30cm), systematic (up
to meter), and many gross errors Attributes: color, accuracy, echo ID,
FWF information, time, etc. Measurement process known Measurement along optical line of
sight
Calibration of point clouds
Integrated calibration and orientation of lidar and image matching point clouds
Where‘s the problem? Lidar point clouds are good within surfaces and for vegetation,
not influenced by (sun) shadows Image matching point clouds are good on edges and have high resolution
physical explanation: aperture size of current laser scanners vs. photogrammetric cameras
Aimcombine the advantages and exploit the differences
Requirementprecise geo-referencing
Standard methodindependent bundle block adjustment and laser scanning strip adjustment
14
Flight block Melk (Austria)
15
Abbey and city of Melkaerial image
Overview map
Flight strips
Mandlburger et al.
Results: data acquisition Melk
17
Dense Image Matching3D RGB point cloud
nframes
Results: data acquisition Melk
18
Airborne LiDAR3D point cloud
Riegl
Elevation differences with standard method
19
Height differences „LiDAR minus DIM“ before integrated strip adjustment / AT
0 100 200 m
Mandlburger et al.
Integration of LiDAR strip adjustment and aerotriangulation
20
Equations• Direct georeferencing• Collinearity• Point to tangent plane ICP
correspondence
Glira et al.
Results: data acquisition Melk
22
Height differences „LiDAR minus DIM“ before integrated strip adjustment / AT
0 100 200 m
Mandlburger et al.
Results: data acquisition Melk
23
Height differences „LiDAR minus DIM“ after integrated strip adjustment / AT
vegetation
vegetation vegetation
Mandlburger et al.
Results: data acquisition Melk
24
Height differences „Lidar minus DIM“ Masked height differences „LiDAR minus DIM“ after hybrid adjustment
0 100 200 m
Mandlburger et al.
DSM generation from LiDAR and DIM
25
LiDARDIM: Default settings DIM: Refinedsettings LiDAR+DIM
Precision (1σ):LiDAR: 2cmDIM: 10cm = 1.67 * GSD
Mandlburger et al.
LiDAR/DIM data properties: vegetation penetration
Vegetation: DIM=top of grass surface, LiDAR=penetrates grass layer Impenetrable surfaces: negligible height differences between DIM and ALS
26
colorized DIM point cloud height difference: LiDAR-DIM
beach volleyball court
soccer fieldrunning trackA BMandlburger et al.
Classification of point clouds
Point cloud classification Principle: use measured and computed features to infer,
which class a point belongs to (features selection?)
28
Echo widthEchoRatio Sigma0 Tran et al.
Classification of point clouds
Point cloud classification Principle: use measured and computed features to infer,
which class a point belongs to (features selection?) Methods: manual decision trees, support vector machines,
random forests, Markov chains, deep convolutional neuralnetworks
Considerations: transferability of rules (across missions, scales, etc.), number of classes, run time, amout of trainingdata, representation of rarely occuring classes, classification accuracy
29
Point cloud
Nor<0.2Ground Y OffGroundN
GreenRatio<0.3393
RoadRoad
Y
Grassland
N Nor<=1.5 Buildings and TreeNLowtree+Car Y
GreenRatio<0.3393 Nor<=5.5
Car LowTree
SmallBuildings and Med Tree
Y
High Buildings and high TreeN
GreenRatio<0.348
Smallbuilding
Y
Medium Tree
N
_omnivariance>1
High Tree
Y
Tree mix building
N
Nor<17
High Tree
Y
High Building
N
Y N
Tran et al.
Change detection in point clouds
A1: classify point clouds of different epochs + compare labels (proximity?) A2: compute geometric point cloud difference + classify changed points Problem 1
change in classand change in geometry can beindependent
Problem 2different dataproperties
Problem 33D changes
30
31
Cut tree
Ground change in height >0.5m
Reconstructed building
New buildings
Unchange
New tree
Ground to another landuse
2005
2011
PC CD envisaged
result
Tran et al.
PC CD Method
Exploit the power of machine learning Features for class and features for change1. measurement process: echo ID, waveform attribute, etc.2. local neighborhood: normal vector, roughness, etc.3. approximate height (simple terrain model)4. change indicators
• distance to nearest point in PC of other epoch• point distribution feature
evaluated at location of epoch 1 pointswith points of epoch 2
32Tran et al.
PC CD Method
Exploit the power of machine learning Features for class and features for change1. measurement process: echo ID, waveform attribute, etc.2. local neighborhood: normal vector, roughness, etc.3. approximate height (simple terrain model)4. change indicators
• distance to nearest point in PC of other epoch• point distribution feature
evaluated at location of epoch 1 pointswith points of epoch 2
Sample training data, learn model (e.g. random forests),
apply model to detect all changes,done (?)
33Tran et al.
PC CD ML
34Tran et al.
PC CD ML
35Tran et al.
36Tran et al.
Conclusions
Many open pointcloud researchquestions
37
References
Tran et al., 2018 (accepted): Classification of image matching point clouds over an urban area. International Journal of Remote Sensing.
Tran et al., 2018: Integrated Change Detection and Classification in Urban Areas Based on Airborne Laser Scanning Point Clouds. Sensors 18.
Mandlburger et al., 2017: Improved Topographic Models via Concurrent Airborne LIDAR and Dense Image Matching. ISPRS Annals IV-2/W4.
Glira et al., 2016: Rigorous Strip adjustment of UAV-based laserscanning data including time-dependent correction of trajectory errors. Photogrammetric Engineering & Remote Sensing 82.
Glira et al., 2015: A Correspondence Framework for ALS Strip Adjustments based on Variants of the ICP. Photogrammetrie, Fernerkundung, Geoinformation 2015.
Pfeifer et al., 2014: OPALS–A framework for Airborne Laser Scanning dataanalysis. Computers, Environment and Urban Systems 45.
Otepka et al., 2013: Georeferenced point clouds: A survey of features and point cloud management. ISPRS International Journal of Geo-Information 2.
38