cosmic microwave background data analysis at nersc
Post on 06-Jan-2016
45 Views
Preview:
DESCRIPTION
TRANSCRIPT
Cosmic Microwave Background Data Analysis At NERSC
Julian Borrillwith
Christopher CantalupoTheodore Kisner
What Is The CMB ?
A snapshot of the Universe when it first became neutral 400,000 years after the Big Bang.
Cosmic - filling all of space.
Microwave - redshifted by the expansion of the Universe from 3000K to 3K.
Background - primordial photons coming from “behind” all astrophysical sources.
Why Do We Care About The CMB ?
The CMB is a unique probe of the very early Universe.Its tiny (1:105-8) fluctuations carry information about - the fundamental parameters of cosmology - ultra-high energy physics beyond the Standard
Model
What Does The CMB Look Like ?
CMB Work At NERSC
• Started in 1997:– 2 separate allocations for Maxima & Boomerang– together 5 users & 30,000 CPU-hours
• Developed into premier world center for CMB analysis:– single allocation shared by O(10) experiments– O(100) users & O(1,000,000) MPP-hrs/year
• Now includes "Big Science" satellite mission– split into two allocations
• mp107 - 13 sub-orbital experiments 40 users & 500,000 MPP-hrs
• planck: Planck satellite 60 users & 2,000,000 MPP-hrs*
The Planck Satellite
•The primary driver for current NERSC CMB work.
•A joint ESA/NASA mission due to launch in the fall of 2008.
•An 18+ month all-sky survey at 9 microwave frequencies from 30 to 857 GHz.
•O(1012) observations, O(108) sky pixels,O(104) spectral multipoles.
Data Management
• Dominated by time-ordered data– O(1-10) TB, O(10,000- 100,000) files
• Each data set must be analyzed as a whole.• Each data analysis needs O(100x) storage.• Each data set may have its own format/distribution.• Each data set must be selectively shared.
• Requires– Pre-fetching & active disk quota management– Efficient & abstracted run-time reading– Project account
Task Management
• Any member of a team must be able to– Access all the data– Access all the general & project-specific codes– Generate and execute standard analyses– Share the results with the team
• Some members of a team must be able to– Control the overall team work-load/-distribution– Manage software versioning and access
• Requires– Project account with individual user certification– Limited capability for most; full capability for
some.– Synchronized data-for-task management
A Framework for CMB Analysis At NERSC
Data Management
Data staging
Run-time IO
Memory
Task Management
User
Project
Critical Components & Issues
• NERSC Global Filesystem– access from Franklin
• Storage Resource Manager– optimal transfer protocols
• Project quotas– separation from UNIX groups
• Project accounts– appropriate queue limits
• User accounts– maintain (unique) accessibility
• Modules– work just fine
Conclusions
• NERSC has developed into the world's leading center for HPC for CMB data analysis– Recognized as such by the recent NASA/NSF/DOE
Weiss report on the future of CMB research.
• This reflects the NERSC resources'– capacity and capability,– accessibility,– long-range development plan.
• Long may it continue !
top related