Concurrent Web Map Cache Server

Zao Liu, Marlon Pierce, Geoffrey Fox

Community Grids Laboratory

Indiana University

Introduction

• Geographical Information Systems combine online dynamic maps and databases.

• Many GIS software packages exist• GIS servers around state of Indiana

– ESRI ArcIMS and ArcMap Server (Marion, Vanderburgh, Hancock, Kosciusco, Huntington, Tippecanoe)

– Autodesk MapGuide (Hamilton, Hendricks, Monroe, Wayne)

– WTH Mapserver™ Web Mapping Application (Fulton, Cass, Daviess, City of Huntingburg) based on several Open Source projects .

• These are not compatible

Map Server Federation• Integrating GIS map servers is not trivial• Our solution: create a virtual map server to act as an agent server

– Translates map requests from generic format to the format expected by the specific map server.

– Provides a common language and programming interface for constructing clients

• The agent server by itself will work but performance is not good– Must wait for slowest server to respond– Failure prone: a county server may not respond at all– Adds additional overhead for combining images

• Combining the agent server with a caching server solves these problems.– Caches images for greater performance

Agent Server Architecture

County Server

Agent server

Caching Server• The agent server runs offline to harvest map images from county

map servers.– Images are stored as tiles.– Tiles at county boundaries may be combined for greater storage and

performance efficiency.• Clients connect to the cache server instead of the agent server.• The cache server constructs the requested image from pre-fetched

tiles.– Inspired by Google Maps approach– Will enable more interactive clients (so-called AJAX programming)

• Image construction may be parallelized/multi-threaded for greater performance.– Potentially takes advantage of new multi-core server architectures from

Sun, Intel, and AMD.

Tiling Example

Agent server requests entire county maps for a particular zoom level and then breaks up into tiles.

Tiling and Caching at County Boundaries

Marion County Hancock County

Bounding box requests across boundaries have many empty tiles.

Removing these empty tiles decreases storage requirements and increases cache server performance

The combined map

Caching and Tiling Layers

• Map servers typically contain base maps and optional layers– Parcel boundaries, roads,

and township boundaries are layers.

• We cache each layer separately.

• Layers and base maps are combined dynamically using Java Advanced Image libraries.– Common techniques

Google Maps – WMS Integration

• Currently Google Map supports attractive street map and aerial photo.

• County map server supports more geographic information in details.

• Issues -- Map Re-projection (Google Map and Sattellite

are based on Mercator projection and WMS are based on EPSG:4326)

-- Caching mechanism (should be the same as Google tiling method)

Map Re-projection

Combine maps (no re-projection) VS Combine maps (re-projection)

Combine Google map with county parcel data

Tradeoffs of Caching

• Cached images must be stored somewhere.

• Currently, three counties (Hancock, Marion, and Cass) are cached at 11 different zoom levels.– Photo images, layers– Takes NN GB of storage

Caching the Entire State

• Takes about 500-600 GB to store the entire state to zoom level 10 this way.– There are 6047804 tiles for zoom levels 0-10, 15131920 tiles for

0-11 levels– There are ~10 layers for each scale

• Aerial photo layer tiles take 60 KB• Other layers (parcels, roads) are much smaller: 36 KB for all

remaining 9 layers per tile• So we need 96KB * 6047804 tiles to store all map data

– This is large but possible.– Current commercial servers hosts like Sun T2000 can have 1 TB

external (RAID) storage.• We can easily spread our caching server over multiple

hosts to store even higher magnification scales.• Efficient tiling storage can save disk space.

Summary of Contributions

• Development of agent server to pre-fetch map images from county map servers.– Stores images as tiles.– Removes redundant/empty tiles.– Supports ESRI and OGC servers

• Development of caching server – Provides a uniform mechanism for clients to interact

with different map servers.– Increases performance and reliability

• Don’t have to go to source map servers for every request.– Will enable more interactive clients

• Google Map-like interaction

Next Steps

• Add more counties– If county uses ESRI or OGC map server,

current agent plugins can be used.

• Formalize programming interface using Web Service standards– WSDL and SOAP

• Investigate scalability and performance issues

Additional Slides

Results & demo• Performance improvement

Architecture

• XMLFetcher

• MapFetcherXMLFetcher

CountyGisServer

Buffer

MapFetcher

DB

Architecture – XMLFetcher

• XMLFetcher– Request info from servers

• fetches XML files not real data

– Save xml into buffer for MapFetcher use• each represents a 1024 * 1024 image

– Send request to several servers for a boundary map• With same coordinates

Architecture – XMLFetcher

Architecture – MapFetcher

• MapFetcher

– Query database

– fetch real data from servers

– Store maps and relate info on local disk

– Tell DBManager to update the database

Tile Reconstruction

• Projection Uniform

• Layer overlap

Tile Reconstruction-- Example

Parcel layer (combined with tiles)

Tile Reconstruction-- Example

• Aerial photo layer