nics rp update teragrid round table march 10, 2011 ryan braby nics hpc operations group lead
TRANSCRIPT
NICS RP Update
TeraGrid Round Table
March 10, 2011
Ryan BrabyNICS HPC Operations Group Lead
Management changes
· Patricia Kovatch is the Interim NICS Project Director.
· Ryan Braby is the new HPC Operations and Technology Integration Group Lead.– Started Jan 1st.– Over 12 years experience in HPC Systems
Administration and Integration.– Experienced with large IBM Power based systems,
BlueGene systems, Linux clusters, and Lustre.
Kraken XT5 Specifications
Compute processor type AMD 2.6 GHz Istanbul-6
Compute cores 112,896
Compute sockets 18,816
Compute nodes 9,408
Memory per node 16 GB (1.33 GB/core)
Total memory 147 TB
Peak system performance 1.17 PF
Interconnect topology 25 x 16 x 24 Torus/Seastar2+
Parallel file system space 3.3 PB (raw) 2.4 PB (usable)
Parallel file system peak performance 30 GB/s
Athena XT4 Specifications
Compute processor type AMD 2.3 GHz Barcelona-4
Compute cores 18,048
Compute sockets 4,512 quad-core
Compute nodes 4,512
Memory per node 4 GB (1 GB/core)
Total memory 17.6 TB
Peak system performance 0.166 PF
Interconnect topology 12 x 16 x 24 Torus/Seastar
Parallel file system space 100 TB (raw) 85 TB (usable)
Parallel file system peak performance 10 GB/s
Nautilus SGI UltraViolet Specs
Compute processor type Intel ~2.0 GHz Nehalem
Compute cores 1024
Compute sockets (nodes) 128 oct-core
Memory per core 4 GB
Total memory 4 TB (NUMA)
Accelerators 16 NVIDIA Fermi GPUs (8 active)
Peak system performance 8.2 TF
Interconnect topology NUMAlink5
Parallel file system space 1 PB (960 TB useable, GPFS)
Parallel file system peak performance 24 GB/s
Kraken Job Mix - Annual
Kraken Job Mix – Jan 2011
14
816
2432
4048
5660
0
1,000,000
2,000,000
3,000,000
4,000,000
5,000,000
6,000,000
7,000,000
63
127
255
511
1023
2047
4095
8191
8256
63
127
255
511
1023
2047
4095
8191
8256Number of Nodes
WallclockHours
Number of NodesCORE-Hours
HPC Ops Report Jan 2011
Kraken and Athena Utilization
%
Month
HPC Ops Report Jan 2011
Feb-10 Mar-10 Apr-10 May-10 Jun-10 Jul-10 Aug-10 Sep-10 Oct-10 Nov-10 Dec-10 Jan-110
10
20
30
40
50
60
70
80
90
100
8784 82
92
9087
96
93
96 95 9796
9394
85
90
9398
95
95
9295 94
93
Kraken XT5 Utilization
Athena Utilization
9
Cycles Provided to TeraGrid
Upcoming Events / Work at NICS
· Kraken upgrade to CLE 2.2 Update 3– March 23rd, 8am to 5pm.– No changes required by users.– Should improve system stability.
· Annual power outage / electrical maintenance– Target is April 2nd, currently planned for 16 hours.