n. doug, q. huang, 2013. chapter 15 geocloud initiative, in spatial cloud computing: a practical...
TRANSCRIPT
N. Doug, Q. Huang, 2013. Chapter 15 GeoCloud Initiative, In Spatial Cloud Computing: a practical approach, edited by C.Yang, Q. Huang, Z. Li, C. Xu, K. Liu, CRC Press: pp. 261-272.
Chapter 15 GeoCloud Initiative
Doug Nebert and Qunying Huang
N. Doug, Q. Huang, 2013. Chapter 15 GeoCloud Initiative, In Spatial Cloud Computing: a practical approach, edited by C.Yang, Q. Huang, Z. Li, C. Xu, K. Liu, CRC Press: pp. 261-272.
Learning Objectives
• Geocloud background
• Geospatial common platform
• Cloud security
• Federal application cloud deployment procedure
N. Doug, Q. Huang, 2013. Chapter 15 GeoCloud Initiative, In Spatial Cloud Computing: a practical approach, edited by C.Yang, Q. Huang, Z. Li, C. Xu, K. Liu, CRC Press: pp. 261-272.
Learning Modules
1. Geocloud background
2. Geocloud architecture
3. Geocloud activities
4. Operational cost in the cloud
5. Lessons learned
6. Discussions
N. Doug, Q. Huang, 2013. Chapter 15 GeoCloud Initiative, In Spatial Cloud Computing: a practical approach, edited by C.Yang, Q. Huang, Z. Li, C. Xu, K. Liu, CRC Press: pp. 261-272.
Background on GeoCloud Sandbox Initiative
• Initiated as an Architecture and Technology Working Group activity in December 2009 with a call to federal agencies to nominate geospatial applications for testing in the Cloud environment for a one-year prototyping process coordinated by FGDC and GSA
• Eleven projects have been nominated by federal agencies as existing projects with existing software suites to be deployed in commercial- or government-hosted Cloud environments
• Two deployment environments (platforms) were drawn from the example projects: Open source service stack on Linux64 and a Commercial service stack on Windows
N. Doug, Q. Huang, 2013. Chapter 15 GeoCloud Initiative, In Spatial Cloud Computing: a practical approach, edited by C.Yang, Q. Huang, Z. Li, C. Xu, K. Liu, CRC Press: pp. 261-272.
Deploying Federal Geospatial Services in the Cloud GeoCloud Goals and Activities
One-year+ project to test and monitor externally-hosted Cloud data and service solutions for the geospatial domain, to support the Geospatial Platform activity
Architectures• Identify requirements-driven solution architectures and platforms
for various sized deployments of geospatial data and services
Cost Models• Document and assess cost models to support scalability, reliability,
and redundancy
Certification• Expedite FISMA (security) certification and accreditation for
agency adoption of packaged solution architectures
Comparisons• Support and collect cost comparison information from agencies for
existing and externally-hosted Cloud solutions• Document lessons learned and best practices
Acquire, compose, document, and deploy reference
platform cloud thatsupport Geospatial Platform standards
Monitor costs, loads, issues and options in support of OMB IT project document guidance
Certify Geospatial Solution Packages to facilitate re-use
Document Best Practices and guides to agencies on adoption
of geospatial Cloud infrastructure
Activities, Outcomes
N. Doug, Q. Huang, 2013. Chapter 15 GeoCloud Initiative, In Spatial Cloud Computing: a practical approach, edited by C.Yang, Q. Huang, Z. Li, C. Xu, K. Liu, CRC Press: pp. 261-272.
GeoCloud Community Platform Value Proposition
Infrastructure as A Service Savings
Hardware Savings Operations Savings Scalability Savings
Platform Savings
Platform building time & effort
Reduced mainten-ance costs
Faster deploy-ment
Cost effective develop-ment /
test
N. Doug, Q. Huang, 2013. Chapter 15 GeoCloud Initiative, In Spatial Cloud Computing: a practical approach, edited by C.Yang, Q. Huang, Z. Li, C. Xu, K. Liu, CRC Press: pp. 261-272.
Platform as a Service Architectural FrameworkDistinguishing Application, Platform and Infrastructure
N. Doug, Q. Huang, 2013. Chapter 15 GeoCloud Initiative, In Spatial Cloud Computing: a practical approach, edited by C.Yang, Q. Huang, Z. Li, C. Xu, K. Liu, CRC Press: pp. 261-272.
Cloud Community Platform Service ActivitiesPrimary Activity Cycle
• Create series of Federal Cloud Computing reference platform prototypes
• Support the wide range of target apps
Prototype
• Ensure platforms work with identified agency applications
Validate
• Document best practices and lessons learned• Document cost models and benchmarks• Bundle the platforms and artifacts• Promulgate to Federal Cloud Community of
Interest
DocumentDocument and
Promulgate
Validate With
Agency Apps
Create Prototype Platforms
N. Doug, Q. Huang, 2013. Chapter 15 GeoCloud Initiative, In Spatial Cloud Computing: a practical approach, edited by C.Yang, Q. Huang, Z. Li, C. Xu, K. Liu, CRC Press: pp. 261-272.
Cloud Community Platform Service Activities Preliminary Platform Architecture
N. Doug, Q. Huang, 2013. Chapter 15 GeoCloud Initiative, In Spatial Cloud Computing: a practical approach, edited by C.Yang, Q. Huang, Z. Li, C. Xu, K. Liu, CRC Press: pp. 261-272.
Deployment options within GeoCloud
• Cost evaluation for each of the initial projects was performed using online ‘calculator’ based on data transfer, storage, CPU, and demand requirements• Most projects could be feasibly hosted in AWS (~$350-500/month)• Some projects were cost-prohibitive in the Cloud due to large data storage
or transfer costs
• Amazon Web Services (EC2) was selected as the primary ‘public’ cloud computing environment for various sizes and numbers of virtual machines (AWS via Apptis is a GSA apps.govIaaS offering on BPA)
• Dell/VMWarevCloud environment was selected for government-hosted cloud infrastructure – at USGS EROS Data Center
N. Doug, Q. Huang, 2013. Chapter 15 GeoCloud Initiative, In Spatial Cloud Computing: a practical approach, edited by C.Yang, Q. Huang, Z. Li, C. Xu, K. Liu, CRC Press: pp. 261-272.
Document and Promulgate
Best practices and lessons learned
Cost models and benchmarks
Bundled platforms, updates, and artifacts
N. Doug, Q. Huang, 2013. Chapter 15 GeoCloud Initiative, In Spatial Cloud Computing: a practical approach, edited by C.Yang, Q. Huang, Z. Li, C. Xu, K. Liu, CRC Press: pp. 261-272.
Project Synopsis
Applications Basic software package NWI Wetlands Mapper AWS, Windows 2008, ArcGIS Server US Census TIGER/Line Downloads AWS, Linux64 (CentOS) Integrated Ocean Observing System Catalog and Viewer (NOAA)
AWS, Linux64 (CentOS), GeoNetwork
NOAA ERDDAP AWS, Linux64 (CentOS), THREDDS EPA Lakes and Ponds AWS, Windows 2008, ArcGIS Server Particles in the Cloud (NOAA) particle tracking computational service for air or water dispersion/ diffusion
AWS, Linux64 (CentOS)
GEOSS Clearinghouse AWS, Linux64 (CentOS) USDA FSA or NRCS data service application
AWS, Windows 2008, ArcGIS Server
N. Doug, Q. Huang, 2013. Chapter 15 GeoCloud Initiative, In Spatial Cloud Computing: a practical approach, edited by C.Yang, Q. Huang, Z. Li, C. Xu, K. Liu, CRC Press: pp. 261-272.
GeoCloud Security
• AWS EC2 Security• garnered approval for hosting 12 public-facing federal
systems • GeoCloud Security Operation
• product-level credential for each project to login• enables the system administrators to provision and release
the VMs, and to configure the network access• Several high level operations were not authorized the
product-level administrators, but only to selected GeoCloud system coordinators
N. Doug, Q. Huang, 2013. Chapter 15 GeoCloud Initiative, In Spatial Cloud Computing: a practical approach, edited by C.Yang, Q. Huang, Z. Li, C. Xu, K. Liu, CRC Press: pp. 261-272.
Operational cost in the cloud
Figure 15. 5. Monthly cost for Tiger/Line and GEOSS Clearinghouse project
N. Doug, Q. Huang, 2013. Chapter 15 GeoCloud Initiative, In Spatial Cloud Computing: a practical approach, edited by C.Yang, Q. Huang, Z. Li, C. Xu, K. Liu, CRC Press: pp. 261-272.
Lessons learned
Security approval process
Software deployment
Time-to-deploy
Failover, redundancy
N. Doug, Q. Huang, 2013. Chapter 15 GeoCloud Initiative, In Spatial Cloud Computing: a practical approach, edited by C.Yang, Q. Huang, Z. Li, C. Xu, K. Liu, CRC Press: pp. 261-272.
Discussion questions
1. What is the GeoCloud initiative?2. What are the benefits to build a common geo-platform?3. What is the GeoCloud architecture?4. Enumerate the GeoCloud activities.5. What are the security considerations? How to achieve security
for federal applications?6. What is operational cost in the cloud for a typical web
application? For data intensive applications?7. What are the lessons learnt?
N. Doug, Q. Huang, 2013. Chapter 15 GeoCloud Initiative, In Spatial Cloud Computing: a practical approach, edited by C.Yang, Q. Huang, Z. Li, C. Xu, K. Liu, CRC Press: pp. 261-272.
References1. AWS Security. 2010. http://aws.amazon.com/security/(accessed on March 13, 2013).2. GEOSS. 2011. GEOSS Clearinghouse Report.
http://www.fgdc.gov/initiatives/geoplatform/geocloud/reports/fgdc-geocloud-project-report-geonetwork.pdf (accessed on March 13, 2013).
3. SDW. 2010. EROS Spatial data warehouse report. http://www.fgdc.gov/initiatives/geoplatform/geocloud/reports/fgdc-geocloud-project-report-eros-sdw.pdf (accessed on March, 13, 2013).
4. U.S. Census Bureau. 2011. Topologically Integrated Geographic Encoding and Referencing (TIGER) TIGER/Line, 2010 Final Report, http://www.fgdc.gov/initiatives/geoplatform/geocloud/reports/fgdc-geocloud-project-report-tiger-2012.pdf (accessed on March, 13, 2013).
5. U.S. Fish and Wildlife Service. 2012. National Wetlands Inventory database metadata: http://www.fws.gov/wetlands/data/metadata/FWS_Wetlands.xml (accessed on March, 13, 2013).
6. Nebert, D. 2010. Deploying Federal Geospatial Services in the Cloud. www.fgdc.gov/ngac/meetings/december-2010/geocloud-briefing.pptx (accessed on March, 13, 2013).
7. Huang, Q., C.Yang, D.Nebert, K.Liu and H.Wu. 2010. Cloud Computing for Geosciences: Deployment of GEOSS Clearinghouse on Amazon’s EC2. In proceedings of ACM SIGSPATIAL International Workshop on High Performance and Distributed Geographic Information Systems (HPDGIS), November 2rd, 2010, San Jose, CA.