integrating open source statistics packages with arcgis · • open source python library for...
TRANSCRIPT
![Page 1: Integrating Open Source Statistics Packages with ArcGIS · • Open Source Python Library for Spatial Analytical Functions ... BSD License. Collaborative Advantages: PySAL and ArcGIS](https://reader031.vdocuments.site/reader031/viewer/2022013022/5fa3c7858e6c8d3bc166747d/html5/thumbnails/1.jpg)
February 24–25, 2016 | Washington, DC
Integrating Open Source Statistics Packages with ArcGIS
Brett Rose
Federal GIS Conference
![Page 2: Integrating Open Source Statistics Packages with ArcGIS · • Open Source Python Library for Spatial Analytical Functions ... BSD License. Collaborative Advantages: PySAL and ArcGIS](https://reader031.vdocuments.site/reader031/viewer/2022013022/5fa3c7858e6c8d3bc166747d/html5/thumbnails/2.jpg)
![Page 3: Integrating Open Source Statistics Packages with ArcGIS · • Open Source Python Library for Spatial Analytical Functions ... BSD License. Collaborative Advantages: PySAL and ArcGIS](https://reader031.vdocuments.site/reader031/viewer/2022013022/5fa3c7858e6c8d3bc166747d/html5/thumbnails/3.jpg)
Why are we here?
Why open source statistics?
![Page 4: Integrating Open Source Statistics Packages with ArcGIS · • Open Source Python Library for Spatial Analytical Functions ... BSD License. Collaborative Advantages: PySAL and ArcGIS](https://reader031.vdocuments.site/reader031/viewer/2022013022/5fa3c7858e6c8d3bc166747d/html5/thumbnails/4.jpg)
Substantive Interest
Mythological choice
Quantitative minds
You’ve asked us
![Page 5: Integrating Open Source Statistics Packages with ArcGIS · • Open Source Python Library for Spatial Analytical Functions ... BSD License. Collaborative Advantages: PySAL and ArcGIS](https://reader031.vdocuments.site/reader031/viewer/2022013022/5fa3c7858e6c8d3bc166747d/html5/thumbnails/5.jpg)
Spatial analytics can mean a lot of thing
![Page 6: Integrating Open Source Statistics Packages with ArcGIS · • Open Source Python Library for Spatial Analytical Functions ... BSD License. Collaborative Advantages: PySAL and ArcGIS](https://reader031.vdocuments.site/reader031/viewer/2022013022/5fa3c7858e6c8d3bc166747d/html5/thumbnails/6.jpg)
With spatial analysis we will
map à to see possible patterns
describe à to improve understanding
measure à to minimize subjectivity
![Page 7: Integrating Open Source Statistics Packages with ArcGIS · • Open Source Python Library for Spatial Analytical Functions ... BSD License. Collaborative Advantages: PySAL and ArcGIS](https://reader031.vdocuments.site/reader031/viewer/2022013022/5fa3c7858e6c8d3bc166747d/html5/thumbnails/7.jpg)
Tools in ArcGIS
![Page 8: Integrating Open Source Statistics Packages with ArcGIS · • Open Source Python Library for Spatial Analytical Functions ... BSD License. Collaborative Advantages: PySAL and ArcGIS](https://reader031.vdocuments.site/reader031/viewer/2022013022/5fa3c7858e6c8d3bc166747d/html5/thumbnails/8.jpg)
Spatial Analyst
# This is map algebraoutRas = Raster("inraster1") + Raster("inraster2")
Mean Slope per Watershed
![Page 9: Integrating Open Source Statistics Packages with ArcGIS · • Open Source Python Library for Spatial Analytical Functions ... BSD License. Collaborative Advantages: PySAL and ArcGIS](https://reader031.vdocuments.site/reader031/viewer/2022013022/5fa3c7858e6c8d3bc166747d/html5/thumbnails/9.jpg)
Geostatistics
![Page 10: Integrating Open Source Statistics Packages with ArcGIS · • Open Source Python Library for Spatial Analytical Functions ... BSD License. Collaborative Advantages: PySAL and ArcGIS](https://reader031.vdocuments.site/reader031/viewer/2022013022/5fa3c7858e6c8d3bc166747d/html5/thumbnails/10.jpg)
Spatial Statistics
![Page 11: Integrating Open Source Statistics Packages with ArcGIS · • Open Source Python Library for Spatial Analytical Functions ... BSD License. Collaborative Advantages: PySAL and ArcGIS](https://reader031.vdocuments.site/reader031/viewer/2022013022/5fa3c7858e6c8d3bc166747d/html5/thumbnails/11.jpg)
Continuing with spatial analytics
python
![Page 12: Integrating Open Source Statistics Packages with ArcGIS · • Open Source Python Library for Spatial Analytical Functions ... BSD License. Collaborative Advantages: PySAL and ArcGIS](https://reader031.vdocuments.site/reader031/viewer/2022013022/5fa3c7858e6c8d3bc166747d/html5/thumbnails/12.jpg)
Spatial Analyst
![Page 13: Integrating Open Source Statistics Packages with ArcGIS · • Open Source Python Library for Spatial Analytical Functions ... BSD License. Collaborative Advantages: PySAL and ArcGIS](https://reader031.vdocuments.site/reader031/viewer/2022013022/5fa3c7858e6c8d3bc166747d/html5/thumbnails/13.jpg)
Spatial & Geo Stats
Spatial Statistics Data Object and Utilities
Data Access Module
![Page 14: Integrating Open Source Statistics Packages with ArcGIS · • Open Source Python Library for Spatial Analytical Functions ... BSD License. Collaborative Advantages: PySAL and ArcGIS](https://reader031.vdocuments.site/reader031/viewer/2022013022/5fa3c7858e6c8d3bc166747d/html5/thumbnails/14.jpg)
Two kinds of “integration”
![Page 15: Integrating Open Source Statistics Packages with ArcGIS · • Open Source Python Library for Spatial Analytical Functions ... BSD License. Collaborative Advantages: PySAL and ArcGIS](https://reader031.vdocuments.site/reader031/viewer/2022013022/5fa3c7858e6c8d3bc166747d/html5/thumbnails/15.jpg)
direct•Numeric/Scientific python modules•Here:
https://wiki.python.org/moin/NumericAndScientific
50+ Modules
Check compatibility
![Page 16: Integrating Open Source Statistics Packages with ArcGIS · • Open Source Python Library for Spatial Analytical Functions ... BSD License. Collaborative Advantages: PySAL and ArcGIS](https://reader031.vdocuments.site/reader031/viewer/2022013022/5fa3c7858e6c8d3bc166747d/html5/thumbnails/16.jpg)
Not direct
• Alternative languages •No python hooks• Incompatible• Python servers as active script and OS•Out of process
![Page 17: Integrating Open Source Statistics Packages with ArcGIS · • Open Source Python Library for Spatial Analytical Functions ... BSD License. Collaborative Advantages: PySAL and ArcGIS](https://reader031.vdocuments.site/reader031/viewer/2022013022/5fa3c7858e6c8d3bc166747d/html5/thumbnails/17.jpg)
got it Now what
![Page 18: Integrating Open Source Statistics Packages with ArcGIS · • Open Source Python Library for Spatial Analytical Functions ... BSD License. Collaborative Advantages: PySAL and ArcGIS](https://reader031.vdocuments.site/reader031/viewer/2022013022/5fa3c7858e6c8d3bc166747d/html5/thumbnails/18.jpg)
pysal• Open Source Python Library for Spatial Analytical Functions• ASU GeoDa Center for Geospatial Analysis and Computation• Luc Anselin• PySpace (GeoDaSpace)
• Sergio Rey• STARS
BSD License
![Page 19: Integrating Open Source Statistics Packages with ArcGIS · • Open Source Python Library for Spatial Analytical Functions ... BSD License. Collaborative Advantages: PySAL and ArcGIS](https://reader031.vdocuments.site/reader031/viewer/2022013022/5fa3c7858e6c8d3bc166747d/html5/thumbnails/19.jpg)
![Page 20: Integrating Open Source Statistics Packages with ArcGIS · • Open Source Python Library for Spatial Analytical Functions ... BSD License. Collaborative Advantages: PySAL and ArcGIS](https://reader031.vdocuments.site/reader031/viewer/2022013022/5fa3c7858e6c8d3bc166747d/html5/thumbnails/20.jpg)
Collaborative Advantages:
PySAL and ArcGIS• Advance Spatial Analysis code
base with novel functions
• E.g. Regionalization, Spatial Econometrics
• Do not have to “reinvent the wheel”
• Experience
• GIS User Interface ~800 GP Tools
• Easy-to-use Script Tool Framework
• Enriched functionalities from ArcGIS arcpy, SSDataObject, SSUtilities, SSReport etc.
• Multiple input/output data format
• Error messages
• Pyharness framework for robust testing
![Page 21: Integrating Open Source Statistics Packages with ArcGIS · • Open Source Python Library for Spatial Analytical Functions ... BSD License. Collaborative Advantages: PySAL and ArcGIS](https://reader031.vdocuments.site/reader031/viewer/2022013022/5fa3c7858e6c8d3bc166747d/html5/thumbnails/21.jpg)
NumPyInput Data
Output Data
SSDataObjectSSUtilities
Environment Settings ProjectionsField Qualification Z/M
Values Bad Records Error/Warning Messages Localization Feature Accounting
Spatial Weights
PySalAnalytical Functions
NumPy
![Page 22: Integrating Open Source Statistics Packages with ArcGIS · • Open Source Python Library for Spatial Analytical Functions ... BSD License. Collaborative Advantages: PySAL and ArcGIS](https://reader031.vdocuments.site/reader031/viewer/2022013022/5fa3c7858e6c8d3bc166747d/html5/thumbnails/22.jpg)
Input Data
SSDataObjectSSUtilities
NumPy
![Page 23: Integrating Open Source Statistics Packages with ArcGIS · • Open Source Python Library for Spatial Analytical Functions ... BSD License. Collaborative Advantages: PySAL and ArcGIS](https://reader031.vdocuments.site/reader031/viewer/2022013022/5fa3c7858e6c8d3bc166747d/html5/thumbnails/23.jpg)
PySAL – ArcGIS Toolbox Demonstration: Regional Income Distributions
![Page 24: Integrating Open Source Statistics Packages with ArcGIS · • Open Source Python Library for Spatial Analytical Functions ... BSD License. Collaborative Advantages: PySAL and ArcGIS](https://reader031.vdocuments.site/reader031/viewer/2022013022/5fa3c7858e6c8d3bc166747d/html5/thumbnails/24.jpg)
R
• R (The R Project for Statistical Computing) • Over 60 CRAN sites across 30+ countries • Its Free GNU GENERAL PUBLIC LICENSE • Base is powerful Statistics, Linear Algebra, Visualization , etc… • Its extendible 1800+ Contributed Extensions • splancs, spatstat, spdep, rgdal, maptools, shapefiles
![Page 25: Integrating Open Source Statistics Packages with ArcGIS · • Open Source Python Library for Spatial Analytical Functions ... BSD License. Collaborative Advantages: PySAL and ArcGIS](https://reader031.vdocuments.site/reader031/viewer/2022013022/5fa3c7858e6c8d3bc166747d/html5/thumbnails/25.jpg)
Indirect Integration Model
• Python and R: “Decoupled”• Used as the core script tool• Hooks into the Operating System to call R• Post-Processor• “Out of Process”
• RPy/RPy2• Compatibility
• win32com• Windows only• Works for other programs as well
![Page 26: Integrating Open Source Statistics Packages with ArcGIS · • Open Source Python Library for Spatial Analytical Functions ... BSD License. Collaborative Advantages: PySAL and ArcGIS](https://reader031.vdocuments.site/reader031/viewer/2022013022/5fa3c7858e6c8d3bc166747d/html5/thumbnails/26.jpg)
Input Data
Enhanced Output
Data
Python
Retrieves Parameters Organizes into R command Executes R command Post-Processing Apply SymbologyApply Projections Report
Input Params
R et al.
Output Data
![Page 27: Integrating Open Source Statistics Packages with ArcGIS · • Open Source Python Library for Spatial Analytical Functions ... BSD License. Collaborative Advantages: PySAL and ArcGIS](https://reader031.vdocuments.site/reader031/viewer/2022013022/5fa3c7858e6c8d3bc166747d/html5/thumbnails/27.jpg)
Python Input Params
![Page 28: Integrating Open Source Statistics Packages with ArcGIS · • Open Source Python Library for Spatial Analytical Functions ... BSD License. Collaborative Advantages: PySAL and ArcGIS](https://reader031.vdocuments.site/reader031/viewer/2022013022/5fa3c7858e6c8d3bc166747d/html5/thumbnails/28.jpg)
R et al.
Output Data Python
Enhanced Output
Data
![Page 29: Integrating Open Source Statistics Packages with ArcGIS · • Open Source Python Library for Spatial Analytical Functions ... BSD License. Collaborative Advantages: PySAL and ArcGIS](https://reader031.vdocuments.site/reader031/viewer/2022013022/5fa3c7858e6c8d3bc166747d/html5/thumbnails/29.jpg)
R – ArcGIS Toolbox Demonstration: Regional Income Distributions
![Page 30: Integrating Open Source Statistics Packages with ArcGIS · • Open Source Python Library for Spatial Analytical Functions ... BSD License. Collaborative Advantages: PySAL and ArcGIS](https://reader031.vdocuments.site/reader031/viewer/2022013022/5fa3c7858e6c8d3bc166747d/html5/thumbnails/30.jpg)
•PySAL•Advanced spatial analytic techniques• Combined with SSDataObject and Utilities • Directly compatible
• Python Harness Implementation • Spatial Econometrics and Spatial Weights Conversion • ESDA, Clustering, Spatial Dynamics etc.
• -BSD
![Page 31: Integrating Open Source Statistics Packages with ArcGIS · • Open Source Python Library for Spatial Analytical Functions ... BSD License. Collaborative Advantages: PySAL and ArcGIS](https://reader031.vdocuments.site/reader031/viewer/2022013022/5fa3c7858e6c8d3bc166747d/html5/thumbnails/31.jpg)
• R • Contains “cutting edge” data analysis techniques
from a wide body of academic and applied fields • Extendible • Indirectly compatible • Direct via RPy/RPy2 and win32com
• GNU • Revolution • esri continues to focus on improving the interaction
in the future
![Page 32: Integrating Open Source Statistics Packages with ArcGIS · • Open Source Python Library for Spatial Analytical Functions ... BSD License. Collaborative Advantages: PySAL and ArcGIS](https://reader031.vdocuments.site/reader031/viewer/2022013022/5fa3c7858e6c8d3bc166747d/html5/thumbnails/32.jpg)
links
• pysal• https://geodacenter.asu.edu/pysal • https://github.com/pysal
• SciPy and NumPy• http://www.scipy.org/
• R• http://www.r-project.org
![Page 33: Integrating Open Source Statistics Packages with ArcGIS · • Open Source Python Library for Spatial Analytical Functions ... BSD License. Collaborative Advantages: PySAL and ArcGIS](https://reader031.vdocuments.site/reader031/viewer/2022013022/5fa3c7858e6c8d3bc166747d/html5/thumbnails/33.jpg)
Try for yourself
http://esri.github.io/
https://github.com/Esri/gis-‐stat-‐analysis-‐py-‐tutor
![Page 34: Integrating Open Source Statistics Packages with ArcGIS · • Open Source Python Library for Spatial Analytical Functions ... BSD License. Collaborative Advantages: PySAL and ArcGIS](https://reader031.vdocuments.site/reader031/viewer/2022013022/5fa3c7858e6c8d3bc166747d/html5/thumbnails/34.jpg)
“Essentially, all models are wrong, but some are useful.” -George E. P. Box