extending arcgis with conda and r - 4326.us€¦ · 01/06/2016 · extending arcgis arcgis is a...
TRANSCRIPT
ExtendingArcGISwithCondaandRShaunWalbridge
https://github.com/esrioceans/oceans-workshop-2016
DownloadPDF
ExtendingArcGIS
ExtendingArcGISArcGISisasystemofrecord.Combinedataandanalysisfrommanyfieldsandintoacommonenvironemnt.Whyextend?Can’tdoitall,wesupportover1000GPtools—enablingintegrationwithotherenvironmentstoextendtheplatform.DiscussedSciPyandwhat’s“inthebox”,nowdiscussthebroaderecosystemofopensource.
Today:RandCondaR-ArcGISBridge
ShortIntroductionDemo
Condaformanagingpackagesandenvironments
IntroductionDemoExercise
R
Why ?Powerfulcoredatastructuresandoperations
Dataframes,functionalprogrammingUnparalleledbreadthofstatisticalroutines
ThedefactolanguageofStatisticiansCRAN:6400packagesforsolvingproblemsVersatileandpowerfulplotting
RDataTypesyou’reusedtoseeing…
Numeric-Integer-Character-Logical-timestampDatatypes
RDataTypesyou’reusedtoseeing…
Numeric-Integer-Character-Logical-timestamp…butothersyouprobablyaren’t:
vector-matrix-data.frame-factor
Datatypes
DataFramesTreatstabular(andmulti-dimensional)dataasalabeled,indexedseriesofobservations.Soundssimple,butisagamechangerovertypicalsoftwarewhichisjustdoing2Dlayout(e.g.Excel)
DataTypes# Create a data frame out of an existing sourcedf.from.csv <- read.csv( "data/growth.csv", header=TRUE)
DataTypes# Create a data frame from scratchquarter <- c(2, 3, 1)person <- c("Goodchild", "Tobler", "Krige")
met.quota <- c(TRUE, FALSE, TRUE)df <- data.frame(person, met.quota, quarter)
DataTypesR> df person met.quota quarter1 Goodchild TRUE 22 Tobler FALSE 33 Krige TRUE 1
0D:SpatialPoints1D:SpatialLines2D:SpatialPolygons3D:Solid4D:Space-time
spTypesEntity+Attributemodel
StatisticalFormulas
DomainspecificlanguageforstatisticsSimilarpropertiesinotherpartsofthelanguage
formodelspecificationconsistency
fit.results <- lm(pollution ~ elevation + rainfall +
caret
R—ArcGISBridge
R—ArcGISBridge
ArcGISdeveloperscancreatetoolsandtoolboxesthatintegrateArcGISandRArcGISuserscanaccessRcodethroughgeoprocessingscriptsRuserscanaccessorganizationsGIS’data,managedintraditionalGISways
https://r-arcgis.github.io
R—ArcGISBridgeStoreyourdatainArcGIS,accessitquicklyinR,returnRobjectsback
toArcGISnativedatatypes(e.g.geodatabasefeatureclasses).
Knowshowtoconvertspatialdatatospobjects.
PackageDocumentation
AccessArcGISfromRStartbyloadingthelibrary,andinitializingconnectiontoArcGIS:
# load the ArcGIS-R bridge librarylibrary(arcgisbinding)# initialize the connection to ArcGIS. Only needed when running directly from R.arc.check_product()
AccessArcGISfromRFirst,selectadatasource(canbeafeatureclass,alayer,oratable):
Then,filterthedatatothesetyouwanttoworkwith(createsin-memorydataframe):
ThiscreatesanArcGISdataframe–lookslikeadataframe,butretainsreferencesbacktothegeometrydata.
input.fc <- arc.open('data.gdb/features')
filtered.df <- arc.select(input.fc, fields=c('fid', 'mean'), where_clause="mean < 100")
AccessArcGISfromRFinishedwithourworkinR,wanttogetthedatabacktoArcGIS.Writeourresultsbacktoanewfeatureclass,witharc.write:
arc.write('data.gdb/new_features', results.df)
BuildingRScriptTools
RArcGISBridgeDemo
DetailsofmodelbasedclusteringanalysisintheRSampleTools
HowToInstallInstallwiththeRbridgeinstallDetailedinstallationinstructions
WhereCanIRunThis?
PackageswithConda
Whypackagemanagement?
Softwareiscomposedofmanysmallercomponents,oftencalledpackagesorlibraries.It’softenbettertoreusecodethatsolvesaproblemwellratherthanrecreatingitBut,sharingcodeisahardproblem.Doyouhavethesamepackagesofthesameversionsasthedeveloperdid?
WhyConda?
ScientificPythoncommunityidentifiedthattherewasagapnotbeingaddressedbythecorePythoninfrastructure,limitingtheirabilitytogetpackagesintothehandsofusers
Industrystandardbuiltbypeoplewhocareaboutthisspace—ContinuumAnalytics
WhyConda?
Itsolvesthehardproblem:
Handlesdependenciesformanylanguages(C,C++,RandofcoursePython)BuiltforPythonfirst,butitreallysolvesamuchbroaderinfrastructuralissue.
Conda
Cross-platform:SimplydeveloprecipesforbuildingandinstallingsoftwareonLinux,OSXandWindows.Opensource:Esriisusingit,youcanuseitinyourownprojectsforothercontexts
Whatcanitinstall?Notjustscientificpackages,caninstalleverythingfrominteractiveenvironmentslike to .Spyder JupyterNotebooks
Environments—CanisolateaPythonenvironment,flexiblymakechangeswithotaffectinginstalledsoftware.Requirements—includeexplicitstateinformation,notjustthepackagename.AlsohandlesplatformsandJupyternotebooks
Wheredopackagescomefrom?
Condapackagescancomefromavarietyoflocations:
manythousandsofpackagesRepositories(e.g.AnacondaCloud,self-hosted)Ondisk
anaconda.org
CondaBasics
Condainfoisthestartingpoint—ittellsyouthestateoftheenvironment.
conda --help
conda info
CondaBasicsconda list
# packages in environment at C:\ArcGIS\bin\Python\envs\arcgispro-py3:#colorama 0.3.7 py35_0 defaultscycler 0.10.0 py35_0 defaultsfuture 0.15.2 py35_0 defaultsmatplotlib 1.5.3 np111py35_0e [arcgispro] esrimpmath 0.19 py35_1 defaultsnetcdf4 1.2.4 py35_0e [arcgispro] esrinose 1.3.7 py35_1 defaultsnumexpr 2.6.1 np111py35_0e [arcgispro] esrinumpy 1.11.2 py35_0e [arcgispro] esripandas 0.19.0 np111py35_0 defaultspip 8.1.2 py35_0 defaultspy 1.4.31 py35_0 defaultspyparsing 2.1.4 py35_0 defaultspypdf2 1.26.0 py_0 esripytest 2.9.2 py35_0 defaultspython 3.5.2 0 defaultspython-dateutil 2.5.3 py35_0 defaultspytz 2016.6.1 py35_0 defaultsrequests 2.11.1 py35_0 defaultsscipy 0.18.1 np111py35_0e [arcgispro] esrisetuptools 27.2.0 py35_1 defaultssympy 1.0 py35_0 defaultswheel 0.29.0 py35_0 defaults
CondaBasicsActivatingenvironments,acoupleways:
UsetheshortcutsManuallyactivatetheenvironment:
cd C:\ArcGIS\bin\Python\Scripts activate arcgispro-py3
CondaBasicsAcollectionofpackagesandPythoninstalliscalledanenvironmentorenv,thebuildingblockformanagingPythonwithCondaCanhavemultipleenvironmentsandseamlesslyswitchbetweenthem
Condavs…Name Means Included?
Conda Thecommanditself ✓Miniconda AminimumsetofPythonpackagesto
buildandrunConda.✓
Anaconda Adistribution200+packagesbuiltwithConda
AnacondaServer
Hostthefullinfrastructureinternally
DeeperDive
HowcanIusethis?WealreadyshipyoutheSciPystack—powerfulandoutoftheboxinallproductsCondacommandandaCondarootPythoninstallNewmodules(e.g.requests),environmentwithProCondaUI
HowcanIusethis?WealreadyshipyoutheSciPystack—powerfulandoutoftheboxinallproductsCondacommandandaCondarootPythoninstallNewmodules(e.g.requests),environmentwithProCondaUIGetpackages,expandyourpossibilityspacePackageyourwork:thisisanopportunitytodistributeit,possiblyincludingcommercialsideaswell.
CanIRunThis?
ArcGISPro1.3+IsthethePythoninstall.
ArcGISPro1.4UIforConda
CanIRunThis?Future:
IntegrationwithplatformHelpmakethingslikeMGETevenbetterShareresultswithorganizationsandothers
CondaHands-onActivity
Hands-onExercise