clip and ship: mapserver for data distribution

Clip and Ship: MapServer for 

Data Distribution  

Frank Warmerdam, Google, Inc.Michael Smith, US Army Corps of Engineers

Clip and Ship

Interactively:• visualization• layer selection• spatial drill down

  Deliver Data:• file format• coordinate system• raw data • bundled up (zip, etc)• immediate delivery

OGC Web Protocols

All are HTTP web service protocols WMS (Web Map Service) • portrayal/visualization oriented • Returns PNG/JPEG/GIF

WFS (Web Feature Service)• Raw GIS vector feature access.• Defaults to returning GML.

 WCS (Web Coverage Service)• Raw Raster data access.• Returns GeoTIFF, etc.

MapServer and GDAL/OGR

GDAL/OGR • multi format raster/vector data access • multi format raster/vector data writing• http://www.gdal.org/

MapServer• Web map server• Supports WMS, WFS and WCS protocols• Uses GDAL/OGR for some data access and encoding• http://www.mapserver.org/

CorpsMap

• MapServer 

• OpenLayers 

• GeoExt 

MapServer Implementation

Map File: • OUTPUTFORMAT declarations for output formats• enable WMS, WFS and WCS • DUMP TRUE for underlying data access• attribute access and type details • declare supported coordinate systems

Metadata to allow Export and Support Projections

WEB   ......   METADATA        "ows_title" "mywfs"        "ows_enable_request" "*"        "ows_srs" "EPSG:4326 EPSG:26915 "        "wfs_getfeature_formatlist" "Shapefile,FileGDB,geojson"   ENDEND 

Including Attributes

• Controlled via gml_include_items and gml_exclude_items

• Separate multiple columns with commas

• Use the special keyword all with the gml_include_items.

Example:METADATA  "gml_include_items" "all"   "gml_exclude_items" "my_sensitive_column,other"   END

• Default behavior is to expose no attributes at all

Attribute Level Controls

• gml_itemname_alias• gml_itemname_width• gml_itemname_precision• gml_itemname_type

o Integer, Real, Character,Date, Boolean or

• "gml_types" "auto"o defines all field types from sourceo OGR, Oracle, PostGIS and Shapefile

• Default is everything is a string!

Geometry Type

gml/ows/wfs_geomtype (Optional)• Geometry, Point, LineString, Polygon, MultiPoint,

MultiLineString, MultiPolygon, GeometryCollection or None

• Also 2.5D version - Point25D, LineString25D, ...

• Defaults to 2D based on layer type

• Used for GML and OGR output

OutputFormat Declaration

Describes a OGR Format including

• layer creation options (LCO:*)

• dataset creation options (DSCO:*)

• formo Simple / Multipart / Zip

•  storageo  Memory / Filesystem / Stream

• filename (default is result.dat)

Example Output Formats

OUTPUTFORMAT  NAME "Shapefile"  DRIVER "OGR/ESRI Shapefile"  FORMATOPTION "STORAGE=memory"  FORMATOPTION "FORM=zip"  FORMATOPTION "FILENAME=%outfile%.zip" END  OUTPUTFORMAT  NAME "FileGDB"  DRIVER "OGR/FileGDB"  FORMATOPTION "STORAGE=filesystem"  FORMATOPTION "FORM=zip"  FORMATOPTION "FILENAME=%outfile%.gdb.zip" END 

More Examples 

OUTPUTFORMAT  NAME "geojson"  DRIVER "OGR/GEOJSON"  MIMETYPE "application/json; subtype=geojson"  FORMATOPTION "STORAGE=stream"  FORMATOPTION "FORM=SIMPLE"END

OUTPUTFORMAT  NAME "OGRGML"  DRIVER "OGR/GML"  FORMATOPTION "STORAGE=memory"  FORMATOPTION "FORM=zip"  FORMATOPTION "FILENAME=%outfile%.gml.zip"END

Even More Examples 

OUTPUTFORMAT  NAME "MIDMIF"  DRIVER "OGR/MapInfo File"  FORMATOPTION "STORAGE=filesystem"  FORMATOPTION "FORM=multipart"  FORMATOPTION "DSCO:FORMAT=MIF"  FORMATOPTION "FILENAME=result.mif"END

OUTPUTFORMAT  NAME "CSV"  DRIVER "OGR/CSV"  MIMETYPE "text/csv"  FORMATOPTION "LCO:GEOMETRY=AS_WKT"  FORMATOPTION "STORAGE=filesystem"  FORMATOPTION "FORM=simple"  FORMATOPTION "FILENAME=result.csv"END

Make Request via WFS

http://server/cgi-bin/mapserv6?map=mymapfile.map&service=wfs&version=1.1.0&request=getFeature&typename=myLayer&outputformat=Shapefile

Make a request for projected data (requires WFS 1.1.0 or later)

http://server/cgi-bin/mapserv6?map=mymapfile.map&service=wfs&version=1.1.0&request=getFeature&typename=myLayer&outputformat=FileGDB&srsname=EPSG:26915

Clip and Ship

http://server/cgi-bin/mapserv6?map=mymapfile.map&service=wfs&version=1.1.0&request=getFeature&typename=myLayer&outputformat=Shapefile&Filter=<Filter> <BBOX>   <PropertyName>      <Name>NAME</Name>   </PropertyName>   <Box srsName='EPSG:4326'>     <Coordinates>-95.844,40.253 -95.384,40.483</coordinates>   </Box>  </BBOX></Filter>

Validation for Filename Substitution

WEB   ....     METADATA     ...     END          VALIDATION         outfile "."         default_outfile "result"     ENDEND

Request with Filename

http://server/cgi-bin/mapserv6?map=mymapfile.map&service=wfs&version=1.1.0&request=getFeature&typename=myLayer&outputformat=Shapefile&outfile=myshapefile

Other Clip and Ship Options

• GeoServer has roughly similar WFS output extensions

• Suggestions from audience?

Lessons Learned

• No temporary files (for some formats)

• Only method to limit download size is maxfeatures

• No method to limit extent of data download

• Every SRS supported for download needs to be enumerated

Conclusion

Docs are available at • OGR Output - http://mapserver.org/output/ogr_output.html

• WFS Server - http://mapserver.org/ogc/wfs_server.html

Supported by MapGears                www.mapgears.com

US Army Corps of Engineers 

CorpsMap Viewer 

OpenLayers / GeoExt / MapServer 6 / Oracle based Web Viewer

Exporting a data layer

Shapefile Export

Zipped Shapefile Generatedhttp://maps.crrel.usace.army.mil:7778/cgi-bin/mapserv6?map=/htdocs/cm2_dev/cm2_test.map &service=wfs&version=1.1.0&request=getFeature&outputformat=Shapefile&outfile=NOAA_Streamgages &Filter=<Filter><BBOX><PropertyName><Name>NAME</Name></PropertyName> <Box srsName='EPSG:4326'><coordinates>-111.626587,36.998444 98.673706,40.992601</coordinates> </Box></BBOX></Filter>&SRSNAME=EPSG:4326&typename=ahpsobsl