best practices for managing and serving lidar and elevation data...
TRANSCRIPT
![Page 1: Best Practices for Managing and Serving Lidar and Elevation Data …proceedings.esri.com/library/userconf/proc16/tech-workshops/tw... · Best Practices for Managing and Serving Lidar](https://reader031.vdocuments.site/reader031/viewer/2022011803/5b77de9c7f8b9ad2498d9647/html5/thumbnails/1.jpg)
Best Practices for Managing and Serving
Lidar and Elevation Data
Cody Benkelman
![Page 2: Best Practices for Managing and Serving Lidar and Elevation Data …proceedings.esri.com/library/userconf/proc16/tech-workshops/tw... · Best Practices for Managing and Serving Lidar](https://reader031.vdocuments.site/reader031/viewer/2022011803/5b77de9c7f8b9ad2498d9647/html5/thumbnails/2.jpg)
Expectations…
• This demo theater presumes an advanced audience…
• Data structures
- LAS & zLAS formats
- LAS Dataset
- Mosaic Dataset
• Longer presentations with further detail:
- Managing Lidar and photogrammetric point clouds
- Wednesday, Room 16 B 3:15 to 4:30 (1845 / 2107)
- Thursday, Room 15 A 1:30 to 2:45 (1845 / 1347)
![Page 3: Best Practices for Managing and Serving Lidar and Elevation Data …proceedings.esri.com/library/userconf/proc16/tech-workshops/tw... · Best Practices for Managing and Serving Lidar](https://reader031.vdocuments.site/reader031/viewer/2022011803/5b77de9c7f8b9ad2498d9647/html5/thumbnails/3.jpg)
Outline
• Workflow
- QA/QC of lidar data
- Creation of elevation surfaces (DTM & DSM)
- Management & sharing of elevation surfaces
- Sharing of lidar point files
• Automated workflows
![Page 4: Best Practices for Managing and Serving Lidar and Elevation Data …proceedings.esri.com/library/userconf/proc16/tech-workshops/tw... · Best Practices for Managing and Serving Lidar](https://reader031.vdocuments.site/reader031/viewer/2022011803/5b77de9c7f8b9ad2498d9647/html5/thumbnails/4.jpg)
Lidar – 3D points and derived 2.5D surfaces
![Page 5: Best Practices for Managing and Serving Lidar and Elevation Data …proceedings.esri.com/library/userconf/proc16/tech-workshops/tw... · Best Practices for Managing and Serving Lidar](https://reader031.vdocuments.site/reader031/viewer/2022011803/5b77de9c7f8b9ad2498d9647/html5/thumbnails/5.jpg)
Lidar QC
![Page 6: Best Practices for Managing and Serving Lidar and Elevation Data …proceedings.esri.com/library/userconf/proc16/tech-workshops/tw... · Best Practices for Managing and Serving Lidar](https://reader031.vdocuments.site/reader031/viewer/2022011803/5b77de9c7f8b9ad2498d9647/html5/thumbnails/6.jpg)
Quality Control Checks
• Class codes
• # of Return values
• Z (height) values
• Is coverage of project area complete?
• Pulse density
• Point density
![Page 7: Best Practices for Managing and Serving Lidar and Elevation Data …proceedings.esri.com/library/userconf/proc16/tech-workshops/tw... · Best Practices for Managing and Serving Lidar](https://reader031.vdocuments.site/reader031/viewer/2022011803/5b77de9c7f8b9ad2498d9647/html5/thumbnails/7.jpg)
Quality Control Checks
• Class codes
- Are they as requested? Accurate?
• # of Return values
- Are First, Second, Third, Fourth returns present?
• Z (height) values
- Is the range reasonable? (“Bird strikes”, points below ground)
• Is coverage of project area complete?
• Pulse density
• Point density
POINT STATISTICS
![Page 8: Best Practices for Managing and Serving Lidar and Elevation Data …proceedings.esri.com/library/userconf/proc16/tech-workshops/tw... · Best Practices for Managing and Serving Lidar](https://reader031.vdocuments.site/reader031/viewer/2022011803/5b77de9c7f8b9ad2498d9647/html5/thumbnails/8.jpg)
Quality Control Checks
• Class codes
- Are they as requested? Accurate?
• # of Return values
- Are First, Second, Third, Fourth returns present?
• Z (height) values
- Is the range reasonable? (“Bird strikes”, points below ground)
• Is coverage of project area complete?
• Pulse density
- # laser pulses sent out, per square meter
• Point density
- “echos” or “returns” received by the sensor
SPATIAL PROCESSING
![Page 9: Best Practices for Managing and Serving Lidar and Elevation Data …proceedings.esri.com/library/userconf/proc16/tech-workshops/tw... · Best Practices for Managing and Serving Lidar](https://reader031.vdocuments.site/reader031/viewer/2022011803/5b77de9c7f8b9ad2498d9647/html5/thumbnails/9.jpg)
Quality Control Part 1 – Data Statistics
• LAS Dataset Properties
![Page 10: Best Practices for Managing and Serving Lidar and Elevation Data …proceedings.esri.com/library/userconf/proc16/tech-workshops/tw... · Best Practices for Managing and Serving Lidar](https://reader031.vdocuments.site/reader031/viewer/2022011803/5b77de9c7f8b9ad2498d9647/html5/thumbnails/10.jpg)
Statistical Reports
• LAS Dataset Properties
![Page 11: Best Practices for Managing and Serving Lidar and Elevation Data …proceedings.esri.com/library/userconf/proc16/tech-workshops/tw... · Best Practices for Managing and Serving Lidar](https://reader031.vdocuments.site/reader031/viewer/2022011803/5b77de9c7f8b9ad2498d9647/html5/thumbnails/11.jpg)
Statistical Reports
• LAS Dataset Properties
![Page 12: Best Practices for Managing and Serving Lidar and Elevation Data …proceedings.esri.com/library/userconf/proc16/tech-workshops/tw... · Best Practices for Managing and Serving Lidar](https://reader031.vdocuments.site/reader031/viewer/2022011803/5b77de9c7f8b9ad2498d9647/html5/thumbnails/12.jpg)
Statistical Reports
• LAS Dataset Properties
![Page 13: Best Practices for Managing and Serving Lidar and Elevation Data …proceedings.esri.com/library/userconf/proc16/tech-workshops/tw... · Best Practices for Managing and Serving Lidar](https://reader031.vdocuments.site/reader031/viewer/2022011803/5b77de9c7f8b9ad2498d9647/html5/thumbnails/13.jpg)
Quality Control Part 2 – Spatial Processing
• Pulse Density
• Summarized per square unit area (meter)
• Output product is a raster (image / pixels)
Example:
4 outgoing pulses per
square meter
![Page 14: Best Practices for Managing and Serving Lidar and Elevation Data …proceedings.esri.com/library/userconf/proc16/tech-workshops/tw... · Best Practices for Managing and Serving Lidar](https://reader031.vdocuments.site/reader031/viewer/2022011803/5b77de9c7f8b9ad2498d9647/html5/thumbnails/14.jpg)
Quality Control Part 2 – Spatial Processing
• Point Density
• Summarized per square unit area (meter)
• Output product is a raster (image / pixels)
Example:
10 return pulses per square
meter
![Page 15: Best Practices for Managing and Serving Lidar and Elevation Data …proceedings.esri.com/library/userconf/proc16/tech-workshops/tw... · Best Practices for Managing and Serving Lidar](https://reader031.vdocuments.site/reader031/viewer/2022011803/5b77de9c7f8b9ad2498d9647/html5/thumbnails/15.jpg)
Quality Control Part 2 – Spatial Processing
• Pulse Density – Loss due to water absorption
• Summarized per square unit area (meter)
• Output product is a raster (image / pixels)
Example:
Fewer return pulses over
water
![Page 16: Best Practices for Managing and Serving Lidar and Elevation Data …proceedings.esri.com/library/userconf/proc16/tech-workshops/tw... · Best Practices for Managing and Serving Lidar](https://reader031.vdocuments.site/reader031/viewer/2022011803/5b77de9c7f8b9ad2498d9647/html5/thumbnails/16.jpg)
![Page 17: Best Practices for Managing and Serving Lidar and Elevation Data …proceedings.esri.com/library/userconf/proc16/tech-workshops/tw... · Best Practices for Managing and Serving Lidar](https://reader031.vdocuments.site/reader031/viewer/2022011803/5b77de9c7f8b9ad2498d9647/html5/thumbnails/17.jpg)
![Page 18: Best Practices for Managing and Serving Lidar and Elevation Data …proceedings.esri.com/library/userconf/proc16/tech-workshops/tw... · Best Practices for Managing and Serving Lidar](https://reader031.vdocuments.site/reader031/viewer/2022011803/5b77de9c7f8b9ad2498d9647/html5/thumbnails/18.jpg)
![Page 19: Best Practices for Managing and Serving Lidar and Elevation Data …proceedings.esri.com/library/userconf/proc16/tech-workshops/tw... · Best Practices for Managing and Serving Lidar](https://reader031.vdocuments.site/reader031/viewer/2022011803/5b77de9c7f8b9ad2498d9647/html5/thumbnails/19.jpg)
Best Practices
• Tiled LAS, v1.1 or higher
• Projected, rearranged, indexed
- zLAS
• File size: 1 – 2 GB or less (<500 MB if not rearranged)
• Keep file I/O local, avoid network
- Study area boundary included as constraint
• Airborne lidar
- Classified (bare earth, non-ground)
- Breaklines for hydro enforcement
• Terrestrial lidar
- RGB & intensity values, classified
* Also applies to photogrammetric point clouds
![Page 20: Best Practices for Managing and Serving Lidar and Elevation Data …proceedings.esri.com/library/userconf/proc16/tech-workshops/tw... · Best Practices for Managing and Serving Lidar](https://reader031.vdocuments.site/reader031/viewer/2022011803/5b77de9c7f8b9ad2498d9647/html5/thumbnails/20.jpg)
DTM & DSM
Creation of Raster Surfaces
![Page 21: Best Practices for Managing and Serving Lidar and Elevation Data …proceedings.esri.com/library/userconf/proc16/tech-workshops/tw... · Best Practices for Managing and Serving Lidar](https://reader031.vdocuments.site/reader031/viewer/2022011803/5b77de9c7f8b9ad2498d9647/html5/thumbnails/21.jpg)
Export raster surfaces from LAS Dataset“Workflow A”
• Most scalable method
• Ensures consistent surfaces for all users
• 5x – 10x data reduction, LAS files do not need to remain on line
• Recommended method for best scalability
• Test before export to define best parameters
• Ensure tiles overlap
• Lidar data may be moved to offline storage
![Page 22: Best Practices for Managing and Serving Lidar and Elevation Data …proceedings.esri.com/library/userconf/proc16/tech-workshops/tw... · Best Practices for Managing and Serving Lidar](https://reader031.vdocuments.site/reader031/viewer/2022011803/5b77de9c7f8b9ad2498d9647/html5/thumbnails/22.jpg)
Tool: LAS Dataset to Tiled Rasters
Download from http://links.esri.com/3dSamples
![Page 23: Best Practices for Managing and Serving Lidar and Elevation Data …proceedings.esri.com/library/userconf/proc16/tech-workshops/tw... · Best Practices for Managing and Serving Lidar](https://reader031.vdocuments.site/reader031/viewer/2022011803/5b77de9c7f8b9ad2498d9647/html5/thumbnails/23.jpg)
Addition of feature constraints“Workflow A”
• Constraints for Hydrology, Breaklines, Model Key points
• DTM only
![Page 24: Best Practices for Managing and Serving Lidar and Elevation Data …proceedings.esri.com/library/userconf/proc16/tech-workshops/tw... · Best Practices for Managing and Serving Lidar](https://reader031.vdocuments.site/reader031/viewer/2022011803/5b77de9c7f8b9ad2498d9647/html5/thumbnails/24.jpg)
Alternative – Rasters generated on-the-fly from lidar points “Workflow B”
• Fastest method to share data
• Can support alternative surfaces (e.g. buildings on DTM, minus trees)
• LAS files should remain on line*
![Page 25: Best Practices for Managing and Serving Lidar and Elevation Data …proceedings.esri.com/library/userconf/proc16/tech-workshops/tw... · Best Practices for Managing and Serving Lidar](https://reader031.vdocuments.site/reader031/viewer/2022011803/5b77de9c7f8b9ad2498d9647/html5/thumbnails/25.jpg)
Managed/Versioned DTM using Terrain Dataset“Workflow C”
• For versioned digital terrain model
• Model points within a geodatabase
![Page 26: Best Practices for Managing and Serving Lidar and Elevation Data …proceedings.esri.com/library/userconf/proc16/tech-workshops/tw... · Best Practices for Managing and Serving Lidar](https://reader031.vdocuments.site/reader031/viewer/2022011803/5b77de9c7f8b9ad2498d9647/html5/thumbnails/26.jpg)
Elevation Data Management
![Page 27: Best Practices for Managing and Serving Lidar and Elevation Data …proceedings.esri.com/library/userconf/proc16/tech-workshops/tw... · Best Practices for Managing and Serving Lidar](https://reader031.vdocuments.site/reader031/viewer/2022011803/5b77de9c7f8b9ad2498d9647/html5/thumbnails/27.jpg)
Source Imagery
SourceMosaic
Datasets
SRTM
LiDAR Project #N
Source Mosaic Datasets – Elevation & Lidar example
LiDAR Project #1
Additional notes:
1) Create overviews on Source Mosaics, then use SRTM (instead of overviews) to fill in “background” elevation values
2) Don’t calculate statistics – it takes too long and statistics for elevation datasets aren’t really meaningful – instead,
use Set Raster Properties to manually insert approximate statistics.
![Page 28: Best Practices for Managing and Serving Lidar and Elevation Data …proceedings.esri.com/library/userconf/proc16/tech-workshops/tw... · Best Practices for Managing and Serving Lidar](https://reader031.vdocuments.site/reader031/viewer/2022011803/5b77de9c7f8b9ad2498d9647/html5/thumbnails/28.jpg)
Source Imagery
SourceMosaic
Datasets
DerivedMosaic Dataset
Combine into Derived Mosaic Dataset
Use TABLE
Raster Type
Advantage: All data available in a
single location
Multi-source, multi-resolution collection of elevation data
![Page 29: Best Practices for Managing and Serving Lidar and Elevation Data …proceedings.esri.com/library/userconf/proc16/tech-workshops/tw... · Best Practices for Managing and Serving Lidar](https://reader031.vdocuments.site/reader031/viewer/2022011803/5b77de9c7f8b9ad2498d9647/html5/thumbnails/29.jpg)
Source Imagery
SourceMosaic
Datasets
DerivedMosaic Dataset
Orthometric Height
f
Single image service with multiple server functions
Example – ArcGIS World Elevation – Server Raster Functions
…many other
functions
Aspect
Ellipsoidal
Height
Slope
Contour
Hillshade
![Page 30: Best Practices for Managing and Serving Lidar and Elevation Data …proceedings.esri.com/library/userconf/proc16/tech-workshops/tw... · Best Practices for Managing and Serving Lidar](https://reader031.vdocuments.site/reader031/viewer/2022011803/5b77de9c7f8b9ad2498d9647/html5/thumbnails/30.jpg)
Source Imagery
SourceMosaic
Datasets
DerivedMosaic Dataset
Orthometric Height
f
Single image service with multiple server functions
Example – ArcGIS World Elevation – Update with new data
…many other
functions
Aspect
Ellipsoidal
Height
Slope
Contour
New data collections added to the central Derived Mosaic appear
immediately in all services
Hillshade
![Page 31: Best Practices for Managing and Serving Lidar and Elevation Data …proceedings.esri.com/library/userconf/proc16/tech-workshops/tw... · Best Practices for Managing and Serving Lidar](https://reader031.vdocuments.site/reader031/viewer/2022011803/5b77de9c7f8b9ad2498d9647/html5/thumbnails/31.jpg)
Lidar-derived surfaces in ArcGIS Online World Elevation Service
Demos
![Page 32: Best Practices for Managing and Serving Lidar and Elevation Data …proceedings.esri.com/library/userconf/proc16/tech-workshops/tw... · Best Practices for Managing and Serving Lidar](https://reader031.vdocuments.site/reader031/viewer/2022011803/5b77de9c7f8b9ad2498d9647/html5/thumbnails/32.jpg)
Support for 3D analysis
![Page 33: Best Practices for Managing and Serving Lidar and Elevation Data …proceedings.esri.com/library/userconf/proc16/tech-workshops/tw... · Best Practices for Managing and Serving Lidar](https://reader031.vdocuments.site/reader031/viewer/2022011803/5b77de9c7f8b9ad2498d9647/html5/thumbnails/33.jpg)
LAS / zLAS files exposed for download – ArcGIS for Server
• Server must have local storage for LAS/zLAS files
• Client = ArcGIS Desktop or custom web client
![Page 34: Best Practices for Managing and Serving Lidar and Elevation Data …proceedings.esri.com/library/userconf/proc16/tech-workshops/tw... · Best Practices for Managing and Serving Lidar](https://reader031.vdocuments.site/reader031/viewer/2022011803/5b77de9c7f8b9ad2498d9647/html5/thumbnails/34.jpg)
LAS / zLAS files exposed for download – Simple download (S3 / FTP)
• Simple cloud storage for LAS/zLAS files, linked to AGOL Feature Service
• Client = browser
![Page 35: Best Practices for Managing and Serving Lidar and Elevation Data …proceedings.esri.com/library/userconf/proc16/tech-workshops/tw... · Best Practices for Managing and Serving Lidar](https://reader031.vdocuments.site/reader031/viewer/2022011803/5b77de9c7f8b9ad2498d9647/html5/thumbnails/35.jpg)
Sharing Geoprocessing Services – Data and Tools in the CloudMove the Processing to the Data, not the Data to the Processing…
• Take advantage of storage and
computing power in the cloud or on a
private server
• Expose Geoprocessing Tools as
services
• Viewshed, Line of sight, Volume
calculations, etc.
• Accessible to Desktop, Web, and
Mobile clients
Geoprocessing
Service
![Page 36: Best Practices for Managing and Serving Lidar and Elevation Data …proceedings.esri.com/library/userconf/proc16/tech-workshops/tw... · Best Practices for Managing and Serving Lidar](https://reader031.vdocuments.site/reader031/viewer/2022011803/5b77de9c7f8b9ad2498d9647/html5/thumbnails/36.jpg)
Geoprocessing Services: Volume calculation
Lidar point download: Sonoma County GIS
Demos
![Page 37: Best Practices for Managing and Serving Lidar and Elevation Data …proceedings.esri.com/library/userconf/proc16/tech-workshops/tw... · Best Practices for Managing and Serving Lidar](https://reader031.vdocuments.site/reader031/viewer/2022011803/5b77de9c7f8b9ad2498d9647/html5/thumbnails/37.jpg)
Automated Workflows – for Repeatability & Scalability
• Simplicity
• Improve Productivity
- Repeatability, Maintainability, Scalability
- Documentation Facilitate QA & QC, Design Review
• Training/Examples
- Encapsulate best practices
- Reusable templates
![Page 38: Best Practices for Managing and Serving Lidar and Elevation Data …proceedings.esri.com/library/userconf/proc16/tech-workshops/tw... · Best Practices for Managing and Serving Lidar](https://reader031.vdocuments.site/reader031/viewer/2022011803/5b77de9c7f8b9ad2498d9647/html5/thumbnails/38.jpg)
Image Management Workflows – Landing pagehttp://resources.arcgis.com
• Overview of Workflows
• Guidebook
- Part of Online Doc
• ArcGIS Online Group
- Gallery of downloadable items
![Page 39: Best Practices for Managing and Serving Lidar and Elevation Data …proceedings.esri.com/library/userconf/proc16/tech-workshops/tw... · Best Practices for Managing and Serving Lidar](https://reader031.vdocuments.site/reader031/viewer/2022011803/5b77de9c7f8b9ad2498d9647/html5/thumbnails/39.jpg)
Resources
• Imagery Resource Center : http://esriurl.com/6005
• Image Management Workflows: http://esriurl.com/6550
• Guidebook in ArcGIS Help: http://esriurl.com/6007
• ArcGIS Online Group: http://esriurl.com/6539
• Recorded Webinar on lidar data management: http://esriurl.com/LTSLidarMgmt
• Optimized LAS tool: http://esriurl.com/zlas
• Tools from 3D Team: http://links.esri.com/3dSamples
• Contact information:
- Cody Benkelman [email protected]
- Lindsay Weitz [email protected]
Managing Lidar
![Page 40: Best Practices for Managing and Serving Lidar and Elevation Data …proceedings.esri.com/library/userconf/proc16/tech-workshops/tw... · Best Practices for Managing and Serving Lidar](https://reader031.vdocuments.site/reader031/viewer/2022011803/5b77de9c7f8b9ad2498d9647/html5/thumbnails/40.jpg)
Managing Lidar