geospatial innovation, science and technology ......understanding where we are headed united nations...
TRANSCRIPT
Charles Brigham
Geospatial Innovation, Science and Technology
Understanding where we are headed
United Nations World Geospatial Information Congress, Deqing, China
0
ArcGIS Is AdvancingMany New and Extensive Capabilities
Where are we
• AI—Artificial Intelligence
• Machine Learning – An AI Branch
- Problems
- Classification
- Regression
- Clustering
- Anomaly Detection
ArcGIS
Overview
• Imagery
• Big Data
• Machine Learning
• Blockchain
• Bringing it all together for the 2030 Agenda
- Federated System for the SDGs
Professional
Imagery &
Geospatial
Analysts
Server
A Comprehensive Imagery PlatformEmpowering users to make informed decisions
System of
Record
Manage and process
all your imagery
System of
Insight
Extract Information
from Imagery
System of
Engagement
Share imagery products and
information to those that need it
Content:
ArcGISOnline
Partners, Org
• Providing Massively Parallel Processing
• . . . Creating New Insights
Parallelized Analyticson Vector and Raster Data
Lightning Strikes
Spatiotemporal Analysis
Big Data GeoAnalytics and Visualization of Massive Data
Solar Suitability
GeoAnalytics
Server
Image
Server
Supporting Single Scenes . . .
. . . and Massive Collections
Imagery Capabilities for Ortho Mapping, Image Management and Analytics
• Powerful Analysis Tools (100+)
• All Major Sensors
• Standard Workflows
Supporting
• Easy to get started, it is “out of the box analytics”
- Install Image Server on nodes -> start Raster Analytic services -> go
- Your own infrastructure, Amazon, Azure
- Scale up & down as required
• Great user experience
- ArcGIS Pro: visual modeler to design simple and complex models
- Portal Map Viewer: Run Raster Analytics Tools
- Python and REST based API for developers
• Results are immediately available in your Web GIS
- No publishing workflow required
- All outputs are written in parallel to ArcGIS distributed storage
Big Data Raster Analytics
Global SRTM 90m
0
100
200
300
400
500
600
700
800
1 2 4 8 16
790
425
252
126
80
Min
ute
s
Raster Analytics Processorsesri virtual machine
• 16GB RAM, 8 cores, NAS storage
13.12 hours
minutes
terrain suitability model• compute slope
• compute aspect
• remap
• overlay
global terrain suitability raster
Global Terrain Suitability
WebGIS (w/ RasterAnalytics) on Amazon
• 8 c3.2xlarge instances (8 vCPUs, 16GB RAM)
Mean Rainfall
Mean Temperature
Elevation
Landcover
30m National Solar Plant Suitability Raster
Raster Analytics
9 minutes
ArcGIS Desktop (ArcMap or ArcGIS Pro)
~4 hours
suitability model
Solar Power Plant Suitability
(foreach) input scene
top of atmosphere
correction
modified soil adjusted
vegetation index
remap to classes
mask
no data
output thematic raster
Esri Web GIS on AWS
Distributed Raster Analytics Cluster
• single node
• AWS c3.8xlarge
• 60GB RAM, 32 cores, 500GB SSD
• 200 Raster Analytics Processors
Infrastructure ProcessingInput Collection Output
Landsat GLS 1990
• 7422 Multispectral Scenes
• S3 storage
Thematic Rasters
• 7422 Thematic Rasters
• Distributed Raster Datastore
50+ scenes per minute
Image Processing: Landsat
ArcGIS ProArcGIS
Server
ArcPy Extension
Big Data
Where Have We Applied This?
Infrastructure Projects
Nation Wide Technology Change Along Highways
Wireless Telco
Phone -> Sensor → Traffic Info
How do we do this?Geoanalytics Server
GeoAnalytics Cluster
ArcGIS Enterprise
GeoAnalytics Server xN
Read
Save
Result
Hosting ServerPortal DataStore
How do we do this?Out of the Box Geoanalytics Tasks
ArcGIS Pro
Courtesy Nathan Shephard, Esri; Rob van Ede, Geological Survey of the Netherlands, Niels van der Vaart, Esri Netherlands; Holger Lipke, Esri Deutschland GMBH
Blockchain for Logistics
Land Management and Addressing
Government
Buildings
ArcGIS Indoors
Land
Records
ArcGIS
3D
Parcels
Planning
Utilities
Indoors
Imagery
Urban
Planning
ArcGIS Urban
Public
Safety
ArcGIS Online
Citizen
Engagement
ArcGIS Hub Construction
AutoCAD
Models
Costs
Materials
Schedules
Engineering
AutoCAD Civil 3D
Detailed
Design
Revit
Conceptual
Design
InfraWorks
Smart Cities with GIS and BIM
Machine
Learning
Deep
Learning
Artificial Intelligence
CNTK TensorFlowTheano
Natural
Language
Processing
Video game
behavioral AI
Robotic
s
Keras
IBM
Watsonscikit-learn
Computer
Vision
ArcGIS has Machine Learning Tools
ArcGIS
Classification
Clustering
Prediction
Machine Learning Tools in ArcGIS
• Maximum Likelihood
Classification
• Random Trees
• Support Vector Machine
Clustering
• Empirical Bayesian Kriging
• Areal Interpolation
• EBK Regression Prediction
• Ordinary Least Squares
Regression and
Exploratory Regression
• Geographically Weighted
Regression
• Spatially Constrained
Multivariate Clustering
• Multivariate Clustering
• Density-based Clustering
• Image Segmentation
• Hot Spot Analysis
• Cluster and Outlier Analysis
• Space Time Pattern Mining
Classification Prediction
Using the known to estimate the unknown
Accurately predict impacts of climate change on local temperature using global climate
model data
Prediction
In ArcGIS: Empirical Bayesian Kriging, Areal Interpolation, EBK Regression Prediction, Ordinary Least Squares
Regression and Exploratory Regression, Geographically Weighted Regression
The grouping of observations based on similarities of values or locations
Given the nearly 50,000 reports of traffic between 5pm and 6pm in Los Angeles (from
Traffic Alerts by Waze), where are traffic zones that can be used to elicit feedback from
current drivers in the area?
Clustering
In ArcGIS: Spatially Constrained Multivariate Clustering, Multivariate Clustering, Density-based Clustering,
Image Segmentation, Hot Spot Analysis, Cluster and Outlier Analysis, Space Time Pattern Mining
The process of deciding to which category an object should be assigned based on a training dataset
Classify impervious surfaces to help effectively prepare for storm and flood events based
on the latest high-resolution imagery
Classification
In ArcGIS: Maximum Likelihood Classification, Random Trees, Support Vector Machine
GeoAI Examples
ETA Prediction Accidents Probability Prediction Smart Road Digitization
High Resolution Land CoverObject Detection from Imagery Object Detection from Videos
AI can help withPrediction
Clustering Anomaly DetectionObject & Feature Detection Land Classification
Vision
Is Enabling a
Sustainable World
GIS
Applied Example: Federated System for the SDGs
SDG Open Data Hubs
Level of
Difficulty
Narrow / Small Broad / Big
Hard
Ea
sy
Organizational Impact
Level of Difficulty
Data Processing &
Management
Story Maps
Analytics
Dashboards
Open Data
Configure vs Customize
The Key is Open: ArcGIS Is Open and Interoperable
Certifications
. . . Successfully Integrated Into Thousands of Systems
AutoCAD
SAP HANA
Adobe Creative Cloud AWS
AzureMS Office
Oracle
SQL Server
Teradata
NetezzaSharePoint
Jupyter Notebook
Power BI
R Python
Revit PostgreSQL
Altibase
Dameng SQLite
Extensible ArchitectureOpen-Source Contributions
Open APIs & SDKsOpen Data Access Open-Source Integration
REST
OPeNDAPWMS
WCS CSW
GML
WFS
KMLWMTSSLD
GeoPackage
SOAP
IFC
LAS
netCDF
SQL
JSON WaterML CityGML
OneGeologyLERC
Shapefiles
Web Scene (I3S)
INSPIREXLSForm
WPSIMDF
Open Standards and Formats
Direct Product Integration
Open Software Architecture
Embeddable
. . . Many
Hadoop
Driven By Participating Member States. . .
. . . Country Owned and Country Led
Evaluate,
Enhance &
Scale
Operate &
Expand
Initial
Operating
Capability
Assess &
Plan
Commitment: A Federated System for the SDGs
Strengthening the monitoring and
Reporting on the 2030 Agenda
Creating a Open SDG Data Hubs
UN SDGHub Network
Web GIS is Enabling a network of SDG Hubs worldwide
Central Statistics Office &
Ordnance Survey Ireland
Ireland
Philippine
Statistics Authority
Philippines
Instituto Nacional de
Estadística y Geografía
(INEGI)
Mexico
A Global SDG Hub … … And a Network of National SDG Hubs
Palestine Central
Bureau of
Statistics
Palestine
What are the technology components?
ArcGIS
. . . Improved Administration, Quality, and Performance
Users
Apps
Enterprise Web GIS in Your InfrastructureArcGIS Server
Online
Improved
• Portal Integration
• Administration Tools
• Vector Tiles
• Smart Mapping
• Spatial Analysis
• Azure Support
• More OCG Support(KML 2.2, WCS 2.0.1, CWS-3.0)
New
• Distributed Portals
• Big Data Extensions
- Vector
- Raster
- Real-TimePortal
Server
Users
Apps
Integrates with On-Premise Services and Systems
Improved
• Smart Mapping
• Formal Metadata
• Administration
New
• Improved Search
• Advanced Analytics
• WFS / WMTS Support
Online Mapping, Analytics, Apps, and Content
Provides Cloud Based GISArcGIS Online
Data, Services, & Apps
Conceptual & Technical Architecture
Desktop
Data Publishers &
Administrators
ArcGIS Hub
ArcGIS Online
Web
Community
APIs
Developers
Hosted Deployment
Mobile
Field Data
Collection
Conceptual & Technical Architecture
ArcGIS Hub
ArcGIS Online
Hybrid Deployment
Portal
ArcGIS Enterprise
Desktop
Data Publishers &
Administrators
Web
Community
APIs
Developers
Mobile
Field Data
Collection
Desktop
Data Publishers &
Administrators
Charles Brigham