experiences & directions of shift/core at cern october 13, 1994
DESCRIPTION
Experiences & directions of SHIFT/CORE at CERN October 13, 1994. Designed in 1990 • fast access to large amounts of data • good tape support • cheap & easy to expand • vendor independent First implementation in operation in Jan 1991. SHIFT. SHIFT Model. CPU Server. CPU Server. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Experiences & directions of SHIFT/CORE at CERN October 13, 1994](https://reader035.vdocuments.site/reader035/viewer/2022070419/56815aa2550346895dc8275e/html5/thumbnails/1.jpg)
F. Hemmer
CERN
HepIx Fall 94
Experiences & directions
of SHIFT/COREat CERN
October 13, 1994
![Page 2: Experiences & directions of SHIFT/CORE at CERN October 13, 1994](https://reader035.vdocuments.site/reader035/viewer/2022070419/56815aa2550346895dc8275e/html5/thumbnails/2.jpg)
F. Hemmer
CERN
HepIx Fall 94
SHIFT
Designed in 1990
• fast access to large amounts of data
• good tape support
• cheap & easy to expand
• vendor independent
First implementation in operation in Jan 1991
![Page 3: Experiences & directions of SHIFT/CORE at CERN October 13, 1994](https://reader035.vdocuments.site/reader035/viewer/2022070419/56815aa2550346895dc8275e/html5/thumbnails/3.jpg)
F. Hemmer
CERN
HepIx Fall 94
SHIFT Model
CPUServerCPU
ServerCPUServerCPU
ServerCPUServerCPU
ServerDisk
ServerDiskServer
TapeServerTape
ServerTapeServer
High Speed network
![Page 4: Experiences & directions of SHIFT/CORE at CERN October 13, 1994](https://reader035.vdocuments.site/reader035/viewer/2022070419/56815aa2550346895dc8275e/html5/thumbnails/4.jpg)
F. Hemmer
CERN
HepIx Fall 94
CORE
• Centrally Operated RISC environment
• Grouped CSF, SHIFT, HOPE,PIAF, TAPES, Network, Racks ...
• CERNVM vs. CORE
![Page 5: Experiences & directions of SHIFT/CORE at CERN October 13, 1994](https://reader035.vdocuments.site/reader035/viewer/2022070419/56815aa2550346895dc8275e/html5/thumbnails/5.jpg)
F. Hemmer
CERN
HepIx Fall 94
1993-1994 Strategy
• Decision to run down the CRAY X/MP
• Decision to run down the IBM 9000
Migrate batch by end of 94
Migrate interactive by end of 95
• Decision to acquire a 64 node SP2
![Page 6: Experiences & directions of SHIFT/CORE at CERN October 13, 1994](https://reader035.vdocuments.site/reader035/viewer/2022070419/56815aa2550346895dc8275e/html5/thumbnails/6.jpg)
F. Hemmer
CERN
HepIx Fall 94
SGI 340 SGI 340SGI 340 SGI 340
SUN
4/330
SUN
4/330
STK 4280
STK 4280
EXABYTE
UltraNet
EXABYTE
JUKEBOX
cisco AGS+IP router
CHEOPS
~250 GBytesSCSI disk
10 GBytesSCSI disk
25 H-P 9000-720s
H-P Apollo DN10040
HOPE
CSF
shift
Simulation
Facilities
Data AnalysisFacility
CrayX-MP/48
cassette tapesand robot
ethernet
CERN site LAN
FDDI
SUN
4/630
STK 4280
STK 4280
STK 4280
STK 4280
CrimsonCrimson
1992
![Page 7: Experiences & directions of SHIFT/CORE at CERN October 13, 1994](https://reader035.vdocuments.site/reader035/viewer/2022070419/56815aa2550346895dc8275e/html5/thumbnails/7.jpg)
F. Hemmer
CERN
HepIx Fall 94
CORE in 1994
Put les core picture
![Page 8: Experiences & directions of SHIFT/CORE at CERN October 13, 1994](https://reader035.vdocuments.site/reader035/viewer/2022070419/56815aa2550346895dc8275e/html5/thumbnails/8.jpg)
F. Hemmer
CERN
HepIx Fall 94
CORE in 1994• CPU server
HP 9000-735/755
SGI Challenge
SGI Power Series
DEC 3000/400,500
IBM RS/6000-370
• Disk servers
SGI Power series
SGI Crimson
DEC 3000-600
IBM RS/6000 970
SUN Sparc 10
• Tape servers
IBM RS/6000-370
Sun 4/330
• Monitors,consoles,YP,...
Sun SPARCstations
TOTAL
# Systems
43
4
1
11
3
1
2
3
1
1
8
3
8
90
# CPUs
43
44
4
11
10
4
2
3
1
1
8
3
8
143
![Page 9: Experiences & directions of SHIFT/CORE at CERN October 13, 1994](https://reader035.vdocuments.site/reader035/viewer/2022070419/56815aa2550346895dc8275e/html5/thumbnails/9.jpg)
F. Hemmer
CERN
HepIx Fall 94
CORE operation
issues• Reliability
• Manpower/machine decreases
• Automation
• Monitoring
• "Standard" tools
• Centralize management
![Page 10: Experiences & directions of SHIFT/CORE at CERN October 13, 1994](https://reader035.vdocuments.site/reader035/viewer/2022070419/56815aa2550346895dc8275e/html5/thumbnails/10.jpg)
F. Hemmer
CERN
HepIx Fall 94
CORE 1994
Here MTBI's per manuf
![Page 11: Experiences & directions of SHIFT/CORE at CERN October 13, 1994](https://reader035.vdocuments.site/reader035/viewer/2022070419/56815aa2550346895dc8275e/html5/thumbnails/11.jpg)
F. Hemmer
CERN
HepIx Fall 94
CORE 1995
• CORE is the infrastructure
operation, network, racks, cables, ...
• Public services
CSF,PIAF,SP2,CS/2
• Public staging pool
• Tape servers
![Page 12: Experiences & directions of SHIFT/CORE at CERN October 13, 1994](https://reader035.vdocuments.site/reader035/viewer/2022070419/56815aa2550346895dc8275e/html5/thumbnails/12.jpg)
F. Hemmer
CERN
HepIx Fall 94
SHIFT CPU 1992
IBM
9000
CSF Cray SHIFT VAX
9000
0
200000
400000
600000
800000
1000000
![Page 13: Experiences & directions of SHIFT/CORE at CERN October 13, 1994](https://reader035.vdocuments.site/reader035/viewer/2022070419/56815aa2550346895dc8275e/html5/thumbnails/13.jpg)
F. Hemmer
CERN
HepIx Fall 94
CORE CPU 1992-94
CERNVM VXCERN CSF SHIFT0
1000000
2000000
3000000
4000000
5000000
![Page 14: Experiences & directions of SHIFT/CORE at CERN October 13, 1994](https://reader035.vdocuments.site/reader035/viewer/2022070419/56815aa2550346895dc8275e/html5/thumbnails/14.jpg)
F. Hemmer
CERN
HepIx Fall 94
SHIFT 1995
here LEP Z0 + Req for 95 + NA 48/49 + LHC
Plus data flows LEP LHC
![Page 15: Experiences & directions of SHIFT/CORE at CERN October 13, 1994](https://reader035.vdocuments.site/reader035/viewer/2022070419/56815aa2550346895dc8275e/html5/thumbnails/15.jpg)
F. Hemmer
CERN
HepIx Fall 94
DISKS 07/94Vendor Model Formatted capacity Number Total size (MB)
IBM OEM 0664CSH IBM OEM 0664M1H 1920 12 23040IBM OEM 0664N1D 1920 228 437760SGI 0664N1D 1920 1 1920MICROP 2112 1001 1 1001HP C2247 1001 10 10010DEC DSP5350 3406 31 105586DEC RZ26-VA 1075 10 10750DEC RZ28-VA 2150 6 12900DEC RZ74 3406 63 214578SEAGATE ST12400 2048 52 106496SEAGATE ST2383N 317 2 634SEAGATE ST41200 990 44 43560SEAGATE ST41600 1331 1 1331SEAGATE ST41650 1351 162 218862SEAGATE ST41651 1350 6 8100SEAGATE ST42100 1812 12 21744SEAGATE ST43400 2778 61 169458SEAGATE ST4766N 669 1 669 Total 704 1392239
![Page 16: Experiences & directions of SHIFT/CORE at CERN October 13, 1994](https://reader035.vdocuments.site/reader035/viewer/2022070419/56815aa2550346895dc8275e/html5/thumbnails/16.jpg)
F. Hemmer
CERN
HepIx Fall 94
Tape mounts week 1-43
1992
4% 4%
75%
17% SHIFT/CRAY
SHIFT/SUN
VM
Others
![Page 17: Experiences & directions of SHIFT/CORE at CERN October 13, 1994](https://reader035.vdocuments.site/reader035/viewer/2022070419/56815aa2550346895dc8275e/html5/thumbnails/17.jpg)
F. Hemmer
CERN
HepIx Fall 94
TAPES in 1994
46%54%
SHIFT
CERNVM
![Page 18: Experiences & directions of SHIFT/CORE at CERN October 13, 1994](https://reader035.vdocuments.site/reader035/viewer/2022070419/56815aa2550346895dc8275e/html5/thumbnails/18.jpg)
F. Hemmer
CERN
HepIx Fall 94
Tapes in 1994
1 7
13
19
25
31
370
50001000015000200002500030000
SHIFT
CERNVM
![Page 19: Experiences & directions of SHIFT/CORE at CERN October 13, 1994](https://reader035.vdocuments.site/reader035/viewer/2022070419/56815aa2550346895dc8275e/html5/thumbnails/19.jpg)
F. Hemmer
CERN
HepIx Fall 94
TAPES 1995
• NTP robots
• 3494 & central data recording
• DLT robot
• Any "reasonable" media support
![Page 20: Experiences & directions of SHIFT/CORE at CERN October 13, 1994](https://reader035.vdocuments.site/reader035/viewer/2022070419/56815aa2550346895dc8275e/html5/thumbnails/20.jpg)
F. Hemmer
CERN
HepIx Fall 94
Network
• UltraNet still the most used
• FDDI used more and more
• 1995 : HIPPi, FCS ...
• Spectrum
![Page 21: Experiences & directions of SHIFT/CORE at CERN October 13, 1994](https://reader035.vdocuments.site/reader035/viewer/2022070419/56815aa2550346895dc8275e/html5/thumbnails/21.jpg)
F. Hemmer
CERN
HepIx Fall 94
UltraNet Bandwith
Oct 92
MB/S
293 294 295 296 297 298 299 300 301 DAY No
0
0.5
1
1.5
2
2.5
3
3.5
![Page 22: Experiences & directions of SHIFT/CORE at CERN October 13, 1994](https://reader035.vdocuments.site/reader035/viewer/2022070419/56815aa2550346895dc8275e/html5/thumbnails/22.jpg)
F. Hemmer
CERN
HepIx Fall 94
Network
Jacek Foils
![Page 23: Experiences & directions of SHIFT/CORE at CERN October 13, 1994](https://reader035.vdocuments.site/reader035/viewer/2022070419/56815aa2550346895dc8275e/html5/thumbnails/23.jpg)
F. Hemmer
CERN
HepIx Fall 94
UltraNet SoftwareUser application Software
Socket Compatibility Library
NFS Sockets
UDP TCPIP
Data LinkDriver
EtherNet UltraNetFDDI
UltraNet Driver
TransportNetwork
Data LinkPhysical Link
Hardware AssistedProtocol Engine
![Page 24: Experiences & directions of SHIFT/CORE at CERN October 13, 1994](https://reader035.vdocuments.site/reader035/viewer/2022070419/56815aa2550346895dc8275e/html5/thumbnails/24.jpg)
F. Hemmer
CERN
HepIx Fall 94
SHIFT Software
• Unix Tape subsystem
multi-user, labels, multi-file, operation
• Fast Remote File Access system
• Remote Tape Copy System
• Disk Pool manager
• Tape Stager
• Clustered NQS Batch System
• Integration with standard I/O packages
Fatmen, Zebra, EPIO
• Network Operation
• Monitoring
![Page 25: Experiences & directions of SHIFT/CORE at CERN October 13, 1994](https://reader035.vdocuments.site/reader035/viewer/2022070419/56815aa2550346895dc8275e/html5/thumbnails/25.jpg)
F. Hemmer
CERN
HepIx Fall 94
Remote Tape Copy System
• select a suitable tape server
• initiates the tape-disk copy
tpread -v I29127 -g SMCF -q 4,6 pathname
tpread -v CUT322 ‘sfget -p opaldst filename‘
![Page 26: Experiences & directions of SHIFT/CORE at CERN October 13, 1994](https://reader035.vdocuments.site/reader035/viewer/2022070419/56815aa2550346895dc8275e/html5/thumbnails/26.jpg)
F. Hemmer
CERN
HepIx Fall 94
RFIO
High performance, reliability (improve on NFS)
• C I/O compatibility
Fortran subroutine interface
• rfio daemon started by open on remote machine
• optimized for specific networks
• asynchronous operation (read ahead)
• optional vector preseek
![Page 27: Experiences & directions of SHIFT/CORE at CERN October 13, 1994](https://reader035.vdocuments.site/reader035/viewer/2022070419/56815aa2550346895dc8275e/html5/thumbnails/27.jpg)
F. Hemmer
CERN
HepIx Fall 94
SW
show what has changed since 1992AFS
![Page 28: Experiences & directions of SHIFT/CORE at CERN October 13, 1994](https://reader035.vdocuments.site/reader035/viewer/2022070419/56815aa2550346895dc8275e/html5/thumbnails/28.jpg)
F. Hemmer
CERN
HepIx Fall 94
Stager
• up to now : dpm + tpread
• lack of robustness faced to system errors
• concurrency not handled
• staging space not handled
![Page 29: Experiences & directions of SHIFT/CORE at CERN October 13, 1994](https://reader035.vdocuments.site/reader035/viewer/2022070419/56815aa2550346895dc8275e/html5/thumbnails/29.jpg)
F. Hemmer
CERN
HepIx Fall 94
Stager architecture
CPUServer
TapeServerDisk
Server
CPUServer
CPUServer
DiskServer
stgcat
TapeServer
TapeServer
![Page 30: Experiences & directions of SHIFT/CORE at CERN October 13, 1994](https://reader035.vdocuments.site/reader035/viewer/2022070419/56815aa2550346895dc8275e/html5/thumbnails/30.jpg)
F. Hemmer
CERN
HepIx Fall 94
Stager functions
• process stage requests
• manage space
- By default purge oldest file when space needed
- if (size < 1024) w=max(atime, mtime)
else w=max(atime,mtime)-(86400*log(size/1024))
• a different algorithm may be provided
![Page 31: Experiences & directions of SHIFT/CORE at CERN October 13, 1994](https://reader035.vdocuments.site/reader035/viewer/2022070419/56815aa2550346895dc8275e/html5/thumbnails/31.jpg)
F. Hemmer
CERN
HepIx Fall 94
Stager commands
• stagein
• stageout/stageput
• stagewrt
• stageclr
• stageqry
![Page 32: Experiences & directions of SHIFT/CORE at CERN October 13, 1994](https://reader035.vdocuments.site/reader035/viewer/2022070419/56815aa2550346895dc8275e/html5/thumbnails/32.jpg)
F. Hemmer
CERN
HepIx Fall 94
Stager future extensions
• Staging of disk files
• Transfer of files already staged on CERNVM
• Access control lists
• Prioritization of requests
• Tape to Tape copy
• Automatic file migration to robots
![Page 33: Experiences & directions of SHIFT/CORE at CERN October 13, 1994](https://reader035.vdocuments.site/reader035/viewer/2022070419/56815aa2550346895dc8275e/html5/thumbnails/33.jpg)
F. Hemmer
CERN
HepIx Fall 94
SHIFT SW Futures
• SHIFT library in CERNLIB
• RFIO review for high speed protocols
• RFIO wide area
• RFIO "gateway" for SP/2
• RFIO per platform tuning
• VMS final port
• RFIO Checkpoint/Restart
• Ports to Irix 6, AIX 4, HP/UX 10