the art of targeted caching: render what you need
TRANSCRIPT
Proprietary & Confidential – Do Not Copy or Distribute Proprietary & Confidential – Do Not Copy or Distribute
Presented by: Josh Foley Quorum Business Solutions
October 25, 2011
The Art of Targeted Caching: Render What You Need
Proprietary & Confidential – Do Not Copy or Distribute
Overview
� Case study y Dynamic vs. Blended Dynamic / Cached services
� Cache management best practices y Scales, coordinate systems, projections y Cache tile migration y Multi-layer cache services
� Cache by Feature Class y Target high-traffic areas y Reduce cache generation time y Reduce storage footprint on servers
Proprietary & Confidential – Do Not Copy or Distribute
What Does it Mean to Cache a Map Service?
� A cached service has a set of map images that have been pre-rendered for rapid display
� Created at pre-determined scale levels
� Tiles stored on the ArcGIS server for fast retrieval
1:250,000
1:500,000
Proprietary & Confidential – Do Not Copy or Distribute Proprietary & Confidential – Do Not Copy or Distribute
Case Study: Un-cached vs. Cached
Proprietary & Confidential – Do Not Copy or Distribute
Map Services
� The following two maps are identical other than the cache status of the Wells, Survey and Base map data
• Agreement – Dynamic • Well – Dynamic • Survey – Dynamic • Base – Dynamic
• Agreement – Dynamic • Well – Cached • Survey – Cached • Base – Cached
Un-cached Cached
Proprietary & Confidential – Do Not Copy or Distribute
Map Services Un-cached Cached
Proprietary & Confidential – Do Not Copy or Distribute
Analysis Un-cached Cached
Proprietary & Confidential – Do Not Copy or Distribute Proprietary & Confidential – Do Not Copy or Distribute
Cache Management Best Practices
Proprietary & Confidential – Do Not Copy or Distribute
Cache Scale Levels
� Blending cached services requires identical scale levels
� Understanding the data you plan to serve to your users is key in determining the scale levels to cache y If you plan to use ArcGIS Online, you should select the
same tile scales to cache
� If your tile’s scale does not match other sources you plan to consume, there will be a compression / expansion of services which will counteract the benefits of caching
Proprietary & Confidential – Do Not Copy or Distribute
Coordinate Systems and Projections
� Blending map services requires a consistent coordinate system / projection y Alignment issues will occur between data in differing
spatial references y Utilize Esri’s native, on the fly projection for cache map
documents
� Set your cache map spatial reference in the data frame properties of the map document you are caching y This will increase the cache generation time, but your
users will not see an impact in performance as they will be pulling the cached tiles
Proprietary & Confidential – Do Not Copy or Distribute
Cache Tile Migration
� Generating caches can be time consuming and managing multiple environments can require multiple caches of the same data y Create the service on the destination y Configure the cache, but do not create the tiles y Migrate the files from the source cache directory
Proprietary & Confidential – Do Not Copy or Distribute
Esri 10 Cache Tile Management
� Export Map Server Cache y Exports tiles from a map cache to a folder on disk drive
- The tiles can either be imported into other caches or they can be accessed from ArcGIS Desktop as a raster dataset, independent from their parent service
y Before running this tool, create the folder on a disk drive that will hold the exported tiles
y Export Map Server Cache lets you choose a specific area or scale from a given cache map service and convert its storage format, while simultaneously exporting the tiles to a location you define
Proprietary & Confidential – Do Not Copy or Distribute
Compact Caching
� Compact is the new Esri 10 format
� Tiles are grouped together into bundles y One bundle can hold up to 16,384 tiles y Reduces the discrepancy between “Size” and “Size on
Disk”
� Benefits y Data mobility y Disk usage y Cache generation throughput y Scalability
Proprietary & Confidential – Do Not Copy or Distribute
Multi-Layer Cache Services
� Cache services are a stored image, therefore it is not possible to turn off individual layers within a service y Build multiple cache services as illustrated in the case
study y Blend the services together in order to view the data as
a single map with the ability to toggle specific services y Create specific services for Wells and Survey to allow
users the ability to turn on / off desired features without losing the Base Culture data underlay
Proprietary & Confidential – Do Not Copy or Distribute Proprietary & Confidential – Do Not Copy or Distribute
Cache by Feature Class
Proprietary & Confidential – Do Not Copy or Distribute
Objective
� Problem y Caching the largest scales consumes the majority of processing
time (final scale ~75% of total time) - This often leaves the scales needed most without cached tiles
y Hogs resources!!
� Tips & Tricks for caching optimization y Target high-traffic areas
� Outcome y Reduces map cache processing time y Reduces storage footprint on servers
Proprietary & Confidential – Do Not Copy or Distribute
Control Model
� A control was utilized to benchmark the processing / memory required for caching the entire state of Wyoming
� Base map service y Cache configured from ArcGIS Online scales
y Cache only for Wyoming
� Five map service instances utilized
Proprietary & Confidential – Do Not Copy or Distribute
Optimized Model
� Dissolved, buffered data inputs create a unique polygon y Agreements – 1 mile buffer y Pipelines – ½ mile buffer y Roads – ½ mile buffer
� Base map service y Cache configured from ArcGIS Online scales y Cache by feature class containing the dissolved,
buffered inputs
� Five map service instances utilized
Proprietary & Confidential – Do Not Copy or Distribute
Cache by Feature Class Toolset
� Allows users to define data inputs
� Creates a feature class for use in the cache
� Allows users to manage map service cache
Proprietary & Confidential – Do Not Copy or Distribute
Buffered Inputs Model
Proprietary & Confidential – Do Not Copy or Distribute
Buffered, Dissolved Inputs
Proprietary & Confidential – Do Not Copy or Distribute
Control Model Analysis
� Elapsed time y 21 hours
� Disk space y 5.15 GB
Proprietary & Confidential – Do Not Copy or Distribute
Optimized Model Analysis
� Elapsed time y 5 hours 47 minutes
� Disk usage y 2.46 GB
Proprietary & Confidential – Do Not Copy or Distribute
Control vs. Optimized – Size on Disk
16 - 1:4508.935525 15 - 1:9017.871050 14 - 1:18035.742100 13 - 1:36071.484201 12 - 1:72142.968401 11 - 1:144285.936802 10 - 1:288571.873604 9 - 1:577143.747209 8 - 1:1154287.494417 7 - 1:2308574.988835 6 - 1:4617149.977669 5 - 1:9234299.955339 4 - 1:18468599.910677 3 - 1:36937199.821354 2 - 1:73874399.642709 1 - 1:147748799.285417
ArcGIS Online scales
Proprietary & Confidential – Do Not Copy or Distribute
Cache at Record Level Changes
� Update Map Server Cache by Feature Class y Checks the feature class to see if the “Cached” field
exists - If it does not exist, it will create the column
y Skips the features that have already been marked as having been cached and picks up where it left off
y Can be scheduled to run as often as data changes y Configurable
- Feature class to target, server name, service name, data frame, scales to cache, update mode and anti-aliasing
Source: http://arcscripts.esri.com/details.asp?dbid=15161
Proprietary & Confidential – Do Not Copy or Distribute
Conclusions
� Dynamic data caching can be achieved with an optimized model
� Ability to cache at all scales
� Models / Python scripts can be configured to business needs
� Flexible enough to target all map services
� Models / Python scripts may be scheduled
� Everyone gets faster maps!
Proprietary & Confidential – Do Not Copy or Distribute Proprietary & Confidential – Do Not Copy or Distribute HOUSTON � DALLAS � CALGARY
Questions?