how to integrate oracle georaster and esri using gdal · 2020-03-20 · • acknowledged oracle...
TRANSCRIPT
7
Futuristic Advances for SMART Airports
OSCARS selected by European Space Agency to provide sustainable SMART Airport Space-based Services:
• Non-invasive Observation and Advanced Management of APRON Ground Operations
• Integrates with existing GIP4 external Airport partner operations and Passenger Journey services
Copyright © 2020, OSCARS-SA and/or its affiliates. All rights reserved.
Visualisation in ArcGIS Pro
Copyright © 2020, OSCARS-SA and/or its affiliates. All rights reserved.
Conclusions
• This solution has been tested and proved to offer good performance. It does not provide a user interface that would allow for browsing and querying the database, like the georaster plugin for QGIS.
• To access a single SDO_GEORASTER object it is necessary to create a small GDAL VRT file pointing to it. Once ArcGIS software have access to the VRT file the raster data can be made available for visualization or any other raster operation.
• There is no way to create new SDO_GEORASTER objects using that method but it can be used to update existing ones. The ArcGIS database connection to Oracle will not going to benefit from that solution, either.
Copyright © 2020, OSCARS-SA and/or its affiliates. All rights reserved.
Oracle GEORASTER support in QGis
• Once upon a time in GIS… in a galaxy not so far away:
• QGIS 2 was supporting oracle georaster format
• But what about QGIS 3?
Copyright © 2020, OSCARS-SA and/or its affiliates. All rights reserved.
Prepare a dataset
Loading GEORASTER :
✓ Creating empty spatial tables
CREATE TABLE RASTER_SAMPLE(
GEORID NUMBER PRIMARY KEY,
SOURCE_FILE VARCHAR2(80),
DESCRIPTION VARCHAR2(32),
GEORASTER SDO_GEORASTER ) ;
CREATE TABLE RASTER_SAMPLE_RDT
OF SDO_RASTER( PRIMARY KEY ( RASTERID,
PYRAMIDLEVEL, BANDBLOCKNUMBER,
ROWBLOCKNUMBER, COLUMNBLOCKNUMBER))
LOB(RASTERBLOCK) STORE AS SECUREFILE (
STORAGE (INITIAL 10G) NOCACHE NOLOGGING );
Loading GEORASTER :
✓ Importing raster data using GDAL (gdalwarp or gdal_translate):
gdalwarp
-of georaster sf1.tif
georaster:SCOTT/TIGER@localhost:1521/PLAYGROUND_PDB,RASTER_SAMPLE,GEORASTER
-co blockxsize=512 -co blockysize=512 -co blockbsize=3
-co interleave=bil
-overwrite
-t_srs EPSG:4326
-co insert="(GEORID, SOURCE_FILE, GEORASTER) values (1, 'sf1.tif’,
sdo_geor.init('RASTER_SAMPLE_RDT'))"
-dstnodata 999
--debug on 2>gdal_trace.log
Copyright © 2020, OSCARS-SA and/or its affiliates. All rights reserved.
Prepare a dataset
Loading GEORASTER :
✓ Creating the metadata
✓ Creating a spatial index
CREATE INDEX raster_sample_sidx ON raster_sample
(georaster.spatialextent) indextype IS
mdsys.spatial_index;
Loading GEORASTER :✓ Building pyramids
DECLARE
v_raster sdo_georaster;
v_start TIMESTAMP;
v_elapsed INTERVAL DAY TO SECOND;
v_load_duration NUMBER;
v_pyramid_duration NUMBER;
BEGIN FOR g IN ( SELECT * FROM us_raster ORDER BY georid ) loop
v_raster := g.georaster;
-- Generate resolution pyramid
v_start := CURRENT_TIMESTAMP;
sdo_geor.generatePyramid(v_raster, 'resampling=NN’);
v_elapsed := CURRENT_TIMESTAMP - v_start;
v_pyramid_duration := EXTRACT (MINUTE FROM v_elapsed) * 60 +
EXTRACT (SECOND FROM v_elapsed);
-- Update the raster object
UPDATE us_raster SET georaster = v_raster
--pyramid_duration = v_pyramid_duration 7
WHERE georid = g.georid;
commit;
END loop;
END;
/
Copyright © 2020, OSCARS-SA and/or its affiliates. All rights reserved.
Open it using QGIS 3
Reading GEORASTER with QGIS3:
Copyright © 2020, OSCARS-SA and/or its affiliates. All rights reserved.
GEORASTER support
Reading GEORASTER with QGIS3:
geor:SCOTT/TIGER@localhost:1521/PLAYGROUND_PDB,RASTER_SAMPLE_RDT,49
Georaster layer
Connection string
RDT table name
RASTERID in the RDT table
Copyright © 2020, OSCARS-SA and/or its affiliates. All rights reserved.
GEORASTER support
Reading GEORASTER with QGIS3:
Copyright © 2020, OSCARS-SA and/or its affiliates. All rights reserved.
GEORASTER support
Reading GEORASTER with QGIS3:
Copyright © 2020, OSCARS-SA and/or its affiliates. All rights reserved.
GEORASTER support
Reading GEORASTER with QGIS3:
Copyright © 2020, OSCARS-SA and/or its affiliates. All rights reserved.
How to find it?
• Coming soon on Oscars website / blogs as dll
• See the last part of slides to build it yourself
• Contact me using [email protected] with Esri plugin as subject.
Copyright © 2020, OSCARS-SA and/or its affiliates. All rights reserved.
• Acknowledged Oracle Spatial Specialist in the GIS Marketplace, with expertise as a Technical Consultant and Trainer
• Established Oscars-SA in 2007 to provide innovative solutions, education and committed personalised consultancy services in the Spatial Domain
• Built close long standing relationships with many leading companies around Europe, over achieving in the delivery of outstanding services
• Inspired the evolution of the GIP4Airports Solution as a complete customizable Intelligent Connected Operations Platform
• Certified by Oracle as a Spatial Implementation, Support, Presales & Sales Specialist
Olivier DuboisCEO
email : [email protected] : https://www.linkedin.com/in/oduboisoscars/
Copyright © 2020, OSCARS-SA and/or its affiliates. All rights reserved.
How to’sOracle GeoRaster
inArcGis Pro
Marc De Saeger
Oscars S.A.
Copyright © 2020, OSCARS-SA and/or its affiliates. All rights reserved.
Requirements
• ArcGIS GDAL_SDK
• GDAL source code
• Oracle InstantClient with SDK
• Visual Studio 2017
• ArcGIS
Copyright © 2020, OSCARS-SA and/or its affiliates. All rights reserved.
Download ArcGIS GDAL_SDK
• Try searching for it on that web page: https://www.arcgis.com/home/search.html?t=content&q=tags%3A%22GDAL%22
• Pick "Custom GDAL ArcGIS Pro 2.4 and 10.7.1"
Copyright © 2020, OSCARS-SA and/or its affiliates. All rights reserved.
Download GDAL source code
• From https://github.com/OSGeo/gdal/tree/release/2.0 click on the green button "Clone or Download" and select "Download ZIP"
• Unzip the package gdal-release-2.0.zip
Copyright © 2020, OSCARS-SA and/or its affiliates. All rights reserved.
Download Oracle InstantClient
• From https://www.oracle.com/database/technologies/instant-client/downloads.html select Windows (x64) and/or (32-bit)
• ArcMap and ArcServer will require 32-bit, ArcPro will require x64.
• From your selection, download the packages Basic and SDK for Oracle 12 at least. That should work with later versions and the other way around might not work.
• It is also useful to download the Sqlplus package so that you have it at command line to check if the connection to Oracle is working.
• Unzip all the packages to the same folder, that they overlap and complement each other
Copyright © 2020, OSCARS-SA and/or its affiliates. All rights reserved.
Building GDAL
• In directory: C:\Dev\GeoRaster\gdal-release-2.0\gdal
• Create a file named nmake.local
• For 64 bits
• For 32 bits
MSVC_VER = 1900
WIN64 = YES
OCI_INCLUDE = -IC:\Dev\GeoRaster\Oracle\64bits\instantclient_12_2\sdk\include
OCI_LIB = C:\Dev\GeoRaster\Oracle\64bits\instantclient_12_2\sdk\lib\msvc\oci.lib
MSVC_VER = 1900
OCI_INCLUDE = -IC:\Dev\GeoRaster\Oracle\32bits\instantclient_12_2\sdk\include
OCI_LIB = C:\Dev\GeoRaster\Oracle\32bits\instantclient_12_2\sdk\lib\msvc\oci.lib
Copyright © 2020, OSCARS-SA and/or its affiliates. All rights reserved.
Building GDAL
• From the windows menu, open the "Visual Studio 2017"/"x64 Native Tools Command Prompt for VS 2017" if you are built for 64-bit or the"Visual Studio 2017"/"x86 Native Tools Command Prompt for VS 2017" if you are building for 32-bit.
• Then execute the following commands:
• To test the GDAL goeraster driver, execute:
• If you know the connection details, the credential and the table, column and unique identifier of a SDO_GEORASTER object you can try also this command:
cd C:\Dev\GeoRaster\gdal-release-2.0\gdal
nmake -f makefile.vc clean
nmake -f makefile.vc
set path=%CD%;%path%
cd apps
gdalinfo --format georaster
gdalinfo geor:USER/PASSWORD@DATABASE,TABLE_NAME,RASTER_COLUMN,COLUMN=VALUE
Copyright © 2020, OSCARS-SA and/or its affiliates. All rights reserved.
Building georaster plugin
• In directory C:\Dev\GeoRaster\gdal-release-2.0\gdal\frmts\georaster
• Create a file named makefile_sdk.vc
Continue on next slide
OCI_LIB = C:\Dev\GeoRaster\Oracle\64bits\instantclient_12_2\sdk\lib\msvc\oci.lib
OCI_INCLUDE = C:\Dev\GeoRaster\Oracle\64bits\instantclient_12_2\sdk\include
PLUGIN_DLL = gdal_GEOR.dll
PLUGIN_PDB = gdal_GEOR.pdb
!IFDEF WIN64
GDAL_LIB = $(GDAL_SDK)\lib\$(VCPREFIX)\x64\gdal201e_i.lib
!ELSE
GDAL_LIB = $(GDAL_SDK)\lib\$(VCPREFIX)\gdal201e_i.lib
!ENDIF
GDAL_INC = $(GDAL_SDK)\include
CFLAGS = /nologo /MD /EHsc /W3 -I$(GDAL_INC) -I$(OCI_INCLUDE) -I..\jpeg\libjpeg -I..\zlib /Zi /Fd$(PLUGIN_PDB)
OBJ = oci_wrapper.obj georaster_dataset.obj georaster_rasterband.obj georaster_wrapper.obj
PLUGIN_DLL = gdal_GEOR.dll
JPEG_LIB = ..\jpeg\libjpeg\libjpeg.lib
default: $(OBJ)
xcopy /D /Y *.obj ..\o
cd ..
Copyright © 2020, OSCARS-SA and/or its affiliates. All rights reserved.
Building georaster plugin
fastread:
cl $(OPTFLAGS) fastread.c $(OCI_INCLUDE) $(OCI_LIB)
clean:
-del *.obj
-del *.dll
-del *.exp
-del *.lib
-del *.manifest
plugin: $(PLUGIN_DLL)
$(PLUGIN_DLL): $(OBJ)
link /dll $(LDEBUG) /out:$(PLUGIN_DLL) $(OBJ) $(GDAL_LIB) $(OCI_LIB) ..\zlib\*.obj ..\jpeg\vsidataio.obj $(JPEG_LIB)
if exist $(PLUGIN_DLL).manifest mt -manifest $(PLUGIN_DLL).manifest -outputresource:$(PLUGIN_DLL);2
.cpp.obj:
$(CC) $(CFLAGS) /c $*.cpp
plugin-install:
-mkdir $(PLUGINDIR)
$(INSTALL) $(PLUGIN_DLL) $(PLUGINDIR)
Copyright © 2020, OSCARS-SA and/or its affiliates. All rights reserved.
Building georaster plugin
• Setup the environment variable GDAL_SDK with the root path of the SDK:
• I you are building for 64-bit you need to setup those two variables:
• If you are building for 32-bit you need to unset this variable
• This other variable is also necessary to identify folder names inside the SDK:
set GDAL_SDK=C:\Dev\GeoRaster\GDAL_SDK_1070_23
set WIN64=YES
set WIN64=
set VCPREFIX=vc14.12
Copyright © 2020, OSCARS-SA and/or its affiliates. All rights reserved.
Building georaster plugin
• In VS2017 prompt window, execute the following commands:
• You are going to get some warnings. Those have been fixed on a later version of GDAL.
• If everything went well, you should have a new file gdal_GEOR.dll on the folder.
• The GDAL georaster driver relays on other drivers to support JPEG200 compression but that is resolved in runtime. You might need to build the OpenJPEG driver as a plugin and install it on ArcGIS, if you want to support SDO_GEORASTER objects compressed with JPEG-F option.
nmake -f makefile_sdk.vc clean
nmake -f makefile_sdk plugin
Copyright © 2020, OSCARS-SA and/or its affiliates. All rights reserved.
Installation
• Create the "gdalplugins" folder under the "bin" folder of your targeted ArcGIS product(s) and copy the newly create DLL file there.
• Ex:copy gdal_GEOR.dll C:\Program Files\ArcGIS\Desktop\bin\gdalplugins
Copyright © 2020, OSCARS-SA and/or its affiliates. All rights reserved.
Testing
• Setup Oracle InstantClientAdd Oracle InstantClient to the system path and create a file tnsname.ora and the environment variable TNS_ADIM pointing to it. Consult your DBA. Try to connect using sqlplus first to make sure that GDAL will be able to connect to Oracle using the same USER/PASSWORD@DATABASE
• CreatePyramids
• Create VRT files using gdal_translate
• Open the file my_first_georaster.vrt on ArcMap or ArcPro
gdaladdo geor:USR/PWD@DB,TABLE,COLUMN,ID=123
gdal_translate -of VRT geor:USR/PWD@DB,TABLE,COLUMN,ID=123 my_first_georaster.vrt
Copyright © 2020, OSCARS-SA and/or its affiliates. All rights reserved.
Visualisation in ArcGIS Pro
Copyright © 2020, OSCARS-SA and/or its affiliates. All rights reserved.
Conclusions
• This solution has been tested and proved to offer good performance. It does not provide a user interface that would allow for browsing and querying the database, like the georaster plugin for QGIS.
• To access a single SDO_GEORASTER object it is necessary to create a small GDAL VRT file pointing to it. Once ArcGIS software have access to the VRT file the raster data can be made available for visualization or any other raster operation.
• There is no way to create new SDO_GEORASTER objects using that method but it can be used to update existing ones. The ArcGIS database connection to Oracle will not going to benefit from that solution, either.
Copyright © 2020, OSCARS-SA and/or its affiliates. All rights reserved.
• Acknowledged Oracle Spatial Specialist in the GIS Marketplace, with expertise as a Technical Consultant and Trainer
• Established Oscars-SA in 2007 to provide innovative solutions, education and committed personalised consultancy services in the Spatial Domain
• Built close long standing relationships with many leading companies around Europe, over achieving in the delivery of outstanding services
• Inspired the evolution of the GIP4Airports Solution as a complete customizable Intelligent Connected Operations Platform
• Certified by Oracle as a Spatial Implementation, Support, Presales & Sales Specialist
Olivier DuboisCEO
email : [email protected] : https://www.linkedin.com/in/oduboisoscars/
Copyright © 2020, OSCARS-SA and/or its affiliates. All rights reserved.