use of foss4g in hybrid systems

Michael Terner, Executive Vice PresidentSeptember, 2014

The role of geospatial open source (FOSS4G) as a component of hybrid systems

© 2 0 1 4 A p p l i e d G e o g r a p h i c s , I n c . S l i d e 2

Terner’s definition of “hybrid system”

A solution that involves more than one technology component, or stack.

Components may have commercial and/or open source origins.

© 2 0 1 4 A p p l i e d G e o g r a p h i c s , I n c . S l i d e 3

Esri blog pieceBy Victoria Kouyoumjian, October, 2011

“It should not be a choice of ‘closed’ or ‘open’ taken in isolation—it’s based on the best business model for you and your organization, and the most appropriate tool that provides Fitness of Purpose…”

© 2 0 1 4 A p p l i e d G e o g r a p h i c s , I n c . S l i d e 4

Do you need to choose just one?Or, is it a best of breed world?

© 2 0 1 4 A p p l i e d G e o g r a p h i c s , I n c . S l i d e 5

But, choosing can be difficult when there are so many choices

• Desktop vs. server vs. cloud

• Open source vs. commercial

• Open source vs. open source

© 2 0 1 4 A p p l i e d G e o g r a p h i c s , I n c . S l i d e 6

In context of FOSS4G, hybrid can be a gateway to Open Source

What is the plan?

• It’s a second database for awhile (it’s OK)

• Train up your staff, they already have the core skills

• Migrate in increments to learn and gain confidence

From yesterday’s presentation on “PostGIS for Managers” by Paul Ramsey, of Boundless

• Bringing in new technology can be difficult– There are sunk costs

– Which make near-term savings more elusive

• Use of Open source does not need to be an all, or nothing decision

© 2 0 1 4 A p p l i e d G e o g r a p h i c s , I n c . S l i d e 7

Why commercial?

• Esri: Legacy systems and sunk costs

– Esri pre-dates Open Source offerings

– Costs in cash and staff know-how

– Really strong professional desktop: cartography/analysis

– Mature business ecosystem

• Google: Unique platform capabilities

– Scalability

– Familiar UI

– All device orientation

© 2 0 1 4 A p p l i e d G e o g r a p h i c s , I n c . S l i d e 8

Four use case examples of hybrid

What?

Why?

How?

© 2 0 1 4 A p p l i e d G e o g r a p h i c s , I n c . S l i d e 9

What?

• Ontario-based media company holding 114 newspapers

• Has an on-line real estate listing

• Listings can be viewed/queried based on neighborhoods (aka “dissemination areas”)

• Neighborhoods are groupings of census geometry

• Neighborhood boundaries are periodically changed by Metroland

– Need for regular polygon aggregation

© 2 0 1 4 A p p l i e d G e o g r a p h i c s , I n c . S l i d e 1 0

Why?Had been using Google as their publication platform

• End-user familiarity

• Curated base map data

• Ability to handle scale

• BUT, Google Maps Engine (GME) does not support polygon aggregation

• So, AppGeo constructed polygon aggregation component using PostGIS

© 2 0 1 4 A p p l i e d G e o g r a p h i c s , I n c . S l i d e 1 1

LinuxNode.js

PostGIS

How?Architecture

Web Server

5. Public use of listings with new

dissemination areas

www.clker.com

1. Metroland initiates process by creating a new “dissemination file” that specifies

aggregation

2. Aggregation file is read and initiates PostGIS polygon aggregation.

3. PostGIS aggregates polygons to create new dissemination

area boundaries

4. New dissemination area boundaries are pushed into GME.

© 2 0 1 4 A p p l i e d G e o g r a p h i c s , I n c . S l i d e 1 2

Crucial Habitat Assessment ToolWhat?

© 2 0 1 4 A p p l i e d G e o g r a p h i c s , I n c . S l i d e 1 3

Crucial Habitat Assessment Tool

• WGA did not specify a technology in their RFP

• RFP required hosting for 1 year; afterwards TBD

• WGA does not have an in-house GIS operation– No in-house infrastructure; no existing licenses

• Relatively simple geo capabilities, but

• Wanted highly customized site– “Unique web destination”

• High performance publishing of 1 large data set

Why?

© 2 0 1 4 A p p l i e d G e o g r a p h i c s , I n c . S l i d e 1 4

How?Hybrid Components

http://westgovchat.org• OpenGeo:

– GeoServer + GeoWebCache for tile making

– PostGIS for geometry and query serving

• Goo– Google Maps API for mapping JavaScript

framework

– Google base maps

• Esri:– Used by states and University of KS for

data management/data aggregation

© 2 0 1 4 A p p l i e d G e o g r a p h i c s , I n c . S l i d e 1 5

How?Architecture

© 2 0 1 4 A p p l i e d G e o g r a p h i c s , I n c . S l i d e 1 6

What?DriveTexas.org

• Public traveler information system

• Chose Google for scalability requirements– Stress tested to 100k queries per

second

• Road conditions data maintained in Esri– Pushed to Google Maps Engine

every 10 minutes

© 2 0 1 4 A p p l i e d G e o g r a p h i c s , I n c . S l i d e 1 7

Why?Component runs every 10 min to synch Esri with Google

Try it at: www.DriveTexas.org (on any device)

© 2 0 1 4 A p p l i e d G e o g r a p h i c s , I n c . S l i d e 1 8

How?Architecture

On-premise Esri DB of road

conditions

1. Custom Esri apps for maintaining road conditions

2. Esri web services publishing current

road conditions

LinuxNode.js3. Component

accesses Esri service to ID conditions changes

and pushes to GME

4. Public use at Google scale

www.clker.com

© 2 0 1 4 A p p l i e d G e o g r a p h i c s , I n c . S l i d e 1 9

Node module for GME access is open

• Node.js tool is built with open components

• Guts of the new AppGeo node code for accessing GME is open source– Used on additional AppGeo projects

(e.g., Metroland)

• Available under MIT license and in AppGeo github organization

© 2 0 1 4 A p p l i e d G e o g r a p h i c s , I n c . S l i d e 2 0

MapGeo migration to Google GMEHosted parcel viewer for local govt.

• Existing tool is built with PostGIS and uses GeoServer/GeoWebCache for tiling

• Moving to Google Maps Engine to ease tiling burden

• PostGIS will remain part of the solution

© 2 0 1 4 A p p l i e d G e o g r a p h i c s , I n c . S l i d e 2 1

Conclusions

• Several models of supplementing commercial solutions with open components

• Cost effective way of extending commercial platforms

• Can be a gateways to organizational use of Open Source

• Share, and share alike

© 2 0 1 4 A p p l i e d G e o g r a p h i c s , I n c . S l i d e 2 2

Questions

?Michael Terner

mgt@appgeo.com