space allocations in nest
DESCRIPTION
Space allocations in NeST. Outline. Overview of NeST Space allocations Implementation Big picture Measurements and Future Work. Overview of NeST. Network storage server designed for the Grid Flexibility Unprivileged, user-level software Multiple protocols Chirp, GridFTP, FTP, HTTP, NFS - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Space allocations in NeST](https://reader035.vdocuments.site/reader035/viewer/2022062723/56813f93550346895daa8486/html5/thumbnails/1.jpg)
Joseph StanleyComputer Sciences DepartmentUniversity of Wisconsin-Madison
[email protected]://www.cs.wisc.edu/condor
Space allocations in NeST
![Page 2: Space allocations in NeST](https://reader035.vdocuments.site/reader035/viewer/2022062723/56813f93550346895daa8486/html5/thumbnails/2.jpg)
www.cs.wisc.edu/condor
Outline
› Overview of NeST
› Space allocations
› Implementation
› Big picture
› Measurements and Future Work
![Page 3: Space allocations in NeST](https://reader035.vdocuments.site/reader035/viewer/2022062723/56813f93550346895daa8486/html5/thumbnails/3.jpg)
www.cs.wisc.edu/condor
Overview of NeST
› Network storage server designed for the Grid
› Flexibility Unprivileged, user-level software Multiple protocols
• Chirp, GridFTP, FTP, HTTP, NFS Multiple concurrency models
• Processes, threads, non-blocking Portable
![Page 4: Space allocations in NeST](https://reader035.vdocuments.site/reader035/viewer/2022062723/56813f93550346895daa8486/html5/thumbnails/4.jpg)
www.cs.wisc.edu/condor
Overview of NeST
› Functionality Exports Unix based file system Space allocation ( Lot )
› Manageability User management – Dynamic
users Security – GSI authentication
![Page 5: Space allocations in NeST](https://reader035.vdocuments.site/reader035/viewer/2022062723/56813f93550346895daa8486/html5/thumbnails/5.jpg)
www.cs.wisc.edu/condor
Dispatcher
Transfer MgrConcurrencies
Control flow
Data flow
Chirp FTP Grid FTP NFS
Common protocol layer
HTTP
Physical network layer
Storage Mgr
Lot Mgr
Physical storage layer
NeST structure
![Page 6: Space allocations in NeST](https://reader035.vdocuments.site/reader035/viewer/2022062723/56813f93550346895daa8486/html5/thumbnails/6.jpg)
www.cs.wisc.edu/condor
Why space allocations ?
› Data generation and storage Data intensive applications Need to allocate space for temporary files
› Data migration Many simultaneous data flows
• Competition for storage space• Possible that all flows end in partial transfers
Need to ensure at least one succeeds
![Page 7: Space allocations in NeST](https://reader035.vdocuments.site/reader035/viewer/2022062723/56813f93550346895daa8486/html5/thumbnails/7.jpg)
www.cs.wisc.edu/condor
Space allocations in NeST
› Lot – abstraction for space allocation
› Create lot for a specified size and duration
› User and group lots
› Guaranteed and best-effort lots
› Hierarchical lots
![Page 8: Space allocations in NeST](https://reader035.vdocuments.site/reader035/viewer/2022062723/56813f93550346895daa8486/html5/thumbnails/8.jpg)
www.cs.wisc.edu/condor
Motivation for hierarchical lot
› SRMs on top of NeSTs
› Client requests streamlined through SRMs
› SRMs create a large lot
› SRMs distribute sub-lots among clients carved from the large lot
![Page 9: Space allocations in NeST](https://reader035.vdocuments.site/reader035/viewer/2022062723/56813f93550346895daa8486/html5/thumbnails/9.jpg)
www.cs.wisc.edu/condor
Lot operations
› Create, Delete, Update› MoveFile
Moves files across lots
› AddUser, RemoveUser Lot level access control List of users allowed to request sub-lots
› Attach / Detach Associates a path to a lot
![Page 10: Space allocations in NeST](https://reader035.vdocuments.site/reader035/viewer/2022062723/56813f93550346895daa8486/html5/thumbnails/10.jpg)
www.cs.wisc.edu/condor
Implementation - two choices
› Rely on the underlying quota system Kernel-assisted lot
› Provide the space guarantee within NeST User-level lot
› A third choice? Modify kernel
![Page 11: Space allocations in NeST](https://reader035.vdocuments.site/reader035/viewer/2022062723/56813f93550346895daa8486/html5/thumbnails/11.jpg)
www.cs.wisc.edu/condor
Dispatcher
Transfer MgrConcurrencies
Control flow
Data flow
Chirp FTP Grid FTP NFS
Common protocol layer
HTTP
Physical network layer
Storage Mgr
Lot Mgr
Physical storage layer
Kernel-assisted lot
Quota system
![Page 12: Space allocations in NeST](https://reader035.vdocuments.site/reader035/viewer/2022062723/56813f93550346895daa8486/html5/thumbnails/12.jpg)
www.cs.wisc.edu/condor
Kernel-assisted lot› Quota system provides the guarantee› Chirp-like protocols pose no challenge
Aware of transfer size
› GridFTP-like protocols ? Transfer size may be unknown Don’t worry at NeST level Quota system watches disk writes
![Page 13: Space allocations in NeST](https://reader035.vdocuments.site/reader035/viewer/2022062723/56813f93550346895daa8486/html5/thumbnails/13.jpg)
www.cs.wisc.edu/condor
Dispatcher
Concurrencies
Control flow
Data flow
Chirp FTP Grid FTP NFS
Common protocol layer
HTTP
Physical network layer
Physical storage layer
Storage Mgr
Lot Mgr
Transfer Mgr
User-level lot
![Page 14: Space allocations in NeST](https://reader035.vdocuments.site/reader035/viewer/2022062723/56813f93550346895daa8486/html5/thumbnails/14.jpg)
www.cs.wisc.edu/condor
User-level lot
› NeST provides the guarantee
› Chirp-like protocols pose no challenge
› GridFTP-like protocols ? Iterative data transfer Transfer manager repeatedly
queries lot manager
![Page 15: Space allocations in NeST](https://reader035.vdocuments.site/reader035/viewer/2022062723/56813f93550346895daa8486/html5/thumbnails/15.jpg)
www.cs.wisc.edu/condor
So which one ?
› Kernel-assisted Pros
Less complexPOSIX access
ConsNeeds root privilegesDepends on quota system – Less portabilityCannot distinguish among different allocations for
the same userDoes not have a hierarchical organization
![Page 16: Space allocations in NeST](https://reader035.vdocuments.site/reader035/viewer/2022062723/56813f93550346895daa8486/html5/thumbnails/16.jpg)
www.cs.wisc.edu/condor
So which one ?
› User-level Pros
Flexible, PortableHierarchical organization is easyEasy to differentiate among lots for the
same user Cons
Difficult to provide POSIX accessComplexity in the transfer and lot
managers
![Page 17: Space allocations in NeST](https://reader035.vdocuments.site/reader035/viewer/2022062723/56813f93550346895daa8486/html5/thumbnails/17.jpg)
www.cs.wisc.edu/condor
Big picture
NeST
Job
DAGMan
SRM
NeST
NeST
Remote site
GridFTP/DiskRouter transfer
StorkRequest allocation
Chirp allocations
Activate 3rd party put
Chirp input
Chirp output
![Page 18: Space allocations in NeST](https://reader035.vdocuments.site/reader035/viewer/2022062723/56813f93550346895daa8486/html5/thumbnails/18.jpg)
www.cs.wisc.edu/condor
Where are the overheads ?
› Lot operations
› Puts
› Recovery
![Page 19: Space allocations in NeST](https://reader035.vdocuments.site/reader035/viewer/2022062723/56813f93550346895daa8486/html5/thumbnails/19.jpg)
www.cs.wisc.edu/condor
Lot operations
![Page 20: Space allocations in NeST](https://reader035.vdocuments.site/reader035/viewer/2022062723/56813f93550346895daa8486/html5/thumbnails/20.jpg)
www.cs.wisc.edu/condor
Puts using Chirp
›With increase in transfer size overhead is amortized
![Page 21: Space allocations in NeST](https://reader035.vdocuments.site/reader035/viewer/2022062723/56813f93550346895daa8486/html5/thumbnails/21.jpg)
www.cs.wisc.edu/condor
Puts using GridFTP
›With increase in transfer size, overheads become comparable
![Page 22: Space allocations in NeST](https://reader035.vdocuments.site/reader035/viewer/2022062723/56813f93550346895daa8486/html5/thumbnails/22.jpg)
www.cs.wisc.edu/condor
Recovery time
›Compaction reduces lot overhead to a manageable level
![Page 23: Space allocations in NeST](https://reader035.vdocuments.site/reader035/viewer/2022062723/56813f93550346895daa8486/html5/thumbnails/23.jpg)
www.cs.wisc.edu/condor
Future Work
› Advance reservations
› Combo - Space and disk bandwidth reservation
› Recovery management Transaction based lot operations
![Page 24: Space allocations in NeST](https://reader035.vdocuments.site/reader035/viewer/2022062723/56813f93550346895daa8486/html5/thumbnails/24.jpg)
www.cs.wisc.edu/condor
Questions ?
› Demo on Wednesday Room 3351, CS building, 9am – noon
› More information available at www.cs.wisc.edu/condor/nest
› People John Bent, Joseph Stanley, Nick LeRoy, Alain
Roy Miron Livny, Andrea Arpaci-Dusseau, Remzi
Arpaci-Dusseau, Arie Shoshani