nasa's movement towards cloud computing
DESCRIPTION
Chris C. Kemp, Chief Technology Officer for IT, NASATRANSCRIPT
National Aeronautics and Space Administration
Cloud Launch
1
The NASA Nebula Project
Chris C. KempChief Technology Officer for IT
National Aeronautics and Space AdministrationNational Aeronautics and Space Administration 2
National Aeronautics and Space AdministrationNational Aeronautics and Space Administration 3
National Aeronautics and Space AdministrationNational Aeronautics and Space Administration 4
• As a result of NASA’s diverse mission and distributed workforce (across centers, labs, and universities), we have a uniquely balkanized – and expensive - IT environment.
• Typical Enterprise IT infrastructure is only utilized to 5-20% of its capacity1
• NASA’s Supercomputers are also only available to the largest projects, with small projects often waiting many weeks in queue… many jobs do not require HPC.
• Thousands of new computers are inefficiently procured and operated each year, continuing the cycle. This process often takes many months and costs more in overhead than the actual value of the computers.
1 Source: http://www.eecs.berkeley.edu/Pubs/TechRpts/2009/EECS-2009-28.pdf
NASA’s IT Challenges
National Aeronautics and Space AdministrationNational Aeronautics and Space Administration 5
National Aeronautics and Space Administration 6
National Aeronautics and Space Administration 7
National Aeronautics and Space Administration 8
National Aeronautics and Space Administration 9
REST, CLI, Web Interfaces
National Aeronautics and Space Administration 10
WISE in the Cloud
124
Nebula Users by Location
7
12
2
5
4
9109
6
11
National Aeronautics and Space AdministrationNational Aeronautics and Space Administration 11
Wide-Field Infrared Survey Explorer (WISE)
Science 8X sky survey• Find the most luminous galaxies in the
universe• Find the closest stars to the sun
Primary Data Products• WISE Image Atlas
10,464 calibrated FITS image sets (4 bands/set), 4kx4k pix @1.375”/pixFormed by combining all single exposures covering Atlas Tile footprintPixel depth-of-coverage, uncertainty maps and metadata for each image
• WISE Source CatalogJ2000 positions, calibrated 4-band photometry, quality and value-added flags and parameters for over 200 million objects in the release area
National Aeronautics and Space AdministrationNational Aeronautics and Space Administration 12
Angular resolution processing.
Processing to resolve and measure source sizes.
#1 1 1 100
1 1 1 2000
1 1 1 100,000
1 80 100 10
1 1 1 100
1 1 1 41,253
Processing to differentiate point-like emission from distributed emission.
Surveying for distributed star formations in our galaxy.
A few large regions can be processed to tremendous depth.
High resolution processing of the entire sky.
WISE in the Cloud
National Aeronautics and Space AdministrationNational Aeronautics and Space Administration 13
Air Traffic Control Algorithm
• An Ames researcher developed an algorithm to produce a useful result from standard ATC data.
• Took 20 hrs on Mac laptop for each airport-day (or 83% of real time)
• Nebula user asked to re-create researcher’s prototype and run a year’s worth of data.
• Initial results: 10241 site-hours (about 1/7 year, first of six phases) in 17 compute hours• or ~85 times faster than the Mac
Laptop• Using 4 tiny instances
National Aeronautics and Space AdministrationNational Aeronautics and Space Administration 14
SERVIR and SPoRT
Science• Numerical Weather Prediction• High-Res Satellite Imagery Processing
Primary Data Products• Rapid Weather Modeling
Shared resources for high resolution processing or large forecast domain.Rapid responses to new events or research opportunities without impacting other resources.
• Satellite ImagesProcess and distribute high resolution images on demand using web-driven, scalable architecture capable of processing large datasets.
National Aeronautics and Space AdministrationNational Aeronautics and Space Administration 15
1 Source: http://www.eecs.berkeley.edu/Pubs/TechRpts/2009/EECS-2009-28.pdf
User Notes
“After having access to Nebula for only a few days, I find that I am already able to accomplish more data-intensive calculations than I can do on any of the local servers we have here, and with no difficulties at all!”
“The system is easy to access and use, and offers a capability that I absolutely need occasionally but for which I could never justify the expense if it were for my needs alone.”
“With the recent addition of a large-RAM instance, I am now able to conduct calculations that could not be done on our project's large server farm. Nebula has provided me with a tool for science data analysis that far surpasses anything that I could envision in a single-user context. NASA Cloud computing may be the way forward for our data-intensive projects in the future, since only a NASA system could provide the necessary reliability and proprietary controls on our data.”
National Aeronautics and Space Administration
Nebula’s Computing Engine, NOVA, is the foundation of OpenStack Compute.
OpenStack is to the datacenter what Linux is to the personal computer.
An open source cloud stack that enables any organization to build large scale private cloud services on commodity hardware.
Standard Apache 2.0 license
National Aeronautics and Space Administration 16
National Aeronautics and Space Administration
600 developers,100 companies,12 countries, 6 months
National Aeronautics and Space Administration 18
Chris C. KempChief Technology Officer for IT
@kemp