workshop details
DESCRIPTION
Online Data Distribution. Field Data Access. A Cloudy View on Computing workshop and CReSIS Field Data Accessibility Jerome Mitchell 1 , Jun Wang 1 , Geoffrey Fox 1 , Linda Hayden 2 Indiana University 1 , Elizabeth City State University 2. WMS. Matlab/GIS. Single User. GIS Cloud Service. - PowerPoint PPT PresentationTRANSCRIPT
This material is based upon work supported by the National Science Foundation under Grant No. ANT-0424589. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author (s) and do not necessarily reflect the views of the National
Science Foundation.
Center for Remote Sensing of Ice SheetsCenter for Remote Sensing of Ice SheetsHeadquarters, University of KansasHeadquarters, University of Kansas
Workshop
DetailsWho: Association of Computer/Information Sciences and Engineering Departments at Minority Institutions (ADMI) faculty/students
Where: Elizabeth City State University (ECSU)
When: June 7 - July 5 2011
What: A Teach-One-Teach-Many approach to cloud computing
Purpose•Introduce ADMI to the basics of the emerging Cloud Computing paradigm
•Understand the computer systems constraints, tradeoffs, and techniques of setting up and using cloud
•Understand how different algorithms can be implemented and executed on cloud frameworks
•Evaluating the performance and identifying bottlenecks when mapping applications to the clouds
A Cloudy View on Computing workshop and CReSIS Field Data Accessibility Jerome Mitchell1, Jun Wang1, Geoffrey Fox1, Linda Hayden2
Indiana University1, Elizabeth City State University2
Schedule
Now I understand Cloud Computing
Now I appreciate why Cloud Computing is important
Now I really understand Cloud Computing!
ParallelProcessing
Map /Reduce
Algorithm Hadoop Twister
Programming Model
Used by
Parallelized by
Apache’s implementation
CGL’s implementation
End of 1st Week
End of 3rd Week
End of 5th Week
Time
Iine
FunctionalProgramming
Compute Resources•FutureGrid
•Virtual machines + virtual networking to create sandboxed moduleso Virtual “Grid” appliances: self-contained, pre-packaged
execution environmentso Group VPNs: simple management of virtual clusters by students
and educators
CReSIS Field Data Accessibility
Current CReSIS Data Organization •CReSIS’s data products website lists
o direct download links for individual files•The data are organized by season
o Seasons are broken into data segments•Data segments are arranged into frames
o Associated data for each frame are stored in different file formats CSV (flight path) MAT (depth sounder data) PDFs (image products)
•File-based data system has no spatial data access support
Spatial Data Accessibility Project•Two main components: Cloud distribution service and special service for PolarGrid field crew.
•Data is supported among multiple spatial databases.
Google EarthGoogle Earth
Matlab/GISMatlab/GIS
GeoServerSpatial Database
GeoServerSpatial Database
GIS Cloud Service
WMS
KML
Online Data Distribution Field Data Access
SpatiaLiteSQLite Database
SpatiaLiteSQLite Database
Field Data Service
Spatial DatabaseVirtual Appliance Spatial DatabaseVirtual Appliance
Data PortalData Portal
Single UserSingle User
Multiple Users(local network)Multiple Users(local network)
Virtual StorageService
Virtual StorageService
Cloud GIS Distribution Service
Google Earth Example
2009 Antarctica Season
Overview of 2009 Flight Paths Data Access for Single Frame
SpatiaLite Databaseo Spatial extension to manages both vector and raster data and supports a rich
set of GIS analysis functions through SQL.
•The data can be directly accessed through GIS software and MATLAB
SpatiaLite Database Example•2009 Antarctic flight path data
o ~ 4 million entries - originally stored as 828 separate files and imported into one SpatiaLite database file
2009 Antarctica Season Vector Data Visual Crossover Analysis for Quality Control (development project)
Flight path data stored as YYYYMMDD_segID_frameID.txtSQLite command to create the segs table:
CREATE TABLE segs ( UTCTime Number, Thickness Number, Elevation Number, FrameID VARCHAR(12), Surface Number, Bottom Number, QualityLevel Integer)
SELECT AddGeometryColumn ('segs','geometry',4326,'POINT',2)*note: geometry: 2 -> xy, (longitude, latitude), 4326 -> WGS84 coordinate system
SpatiaLite: MATLAB Direct AccessMksqlite package: a MEX-DLL to access SQLite databases from MATLAB http://mksqlite.berlios.de/Add this flag to build.m to enable SQLite to load SpatiaLite as an extension: -DSQLITE_ENABLE_LOAD_EXTENSION=1Testing in MATLAB:dbid = mksqlite(0,'open', ‘test.sqlite' )sql = ['SELECT load_extension(''', path_to_spatialite, ''')'];mksqlite(dbid, sql) % load extensionmksqlite(dbid, 'SELECT sqlite_version()')mksqlite(dbid, 'SELECT spatialite_version()')mksqlite(dbid, 'SELECT X(geometry) as lon, Y(geometry) as lat from segs where FrameID=2009101601001'); mksqlite(dbid, 'close')
ReferencesPolarGrid Data Products: https://www.cresis.ku.edu/dataSpatiaLite: http://www.gaia-gis.it/spatialite/Quantum GIS: http://www.qgis.org/