introduction to xfs hepsysman 5 th december 2005
DESCRIPTION
Introduction to XFS HEPSYSMAN 5 th December 2005. What exactly is a filesystem ?. Organization: Large disk space to store small data items Use files and directories In UNIX everything is a stream of bytes: More complex file definition in other OS’s - PowerPoint PPT PresentationTRANSCRIPT
Jonathan Wheeler
E-Science Centre, RAL
Introduction to XFS
HEPSYSMAN
5th December 2005
Jonathan Wheeler
e-Science Centre, RAL
What exactly is a filesystem ?
Organization:• Large disk space to store small data items• Use files and directories
In UNIX everything is a stream of bytes:• More complex file definition in other OS’s• Devices are also presented in the filesystem• Linux uses virtual filesystems, e.g. /proc
Jonathan Wheeler
e-Science Centre, RAL
A little bit of background
Three generation of UNIX filesystems:
a) v7 / sysv / coherent / minix
b) ffs / ext2
c) vxfs / ext3 / jfs / xfs / reiserfs
Jonathan Wheeler
e-Science Centre, RAL
Features of 3rd generation UNIX filesystems
a) Intent logging / journaling
b) Flexible metadata structures
c) Dynamic inode allocations
d) Extents
Jonathan Wheeler
e-Science Centre, RAL
History of XFS
Developed by Silicon Graphics (now SGI) for use on IRIX:
• 1993 - original design• 1994 - available for IRIX v 5.3• 2000 - released as Open Source• 2001/2002 – some Linux distributions with
native XFS support• 2003 – XFS support in 2.4 kernel
Jonathan Wheeler
e-Science Centre, RAL
XFS features (1)
Journaling:
• but only metadata updates, not data updates.
Fast Transactions:
• uses efficient B+ tree structures for fast searches and rapid space allocation
Jonathan Wheeler
e-Science Centre, RAL
XFS Features (2)
64-bit file system:a) 64bit variables for global counters and disk addresses
b) Large theoretical maximum filesystem sizei. For Linux 2.4: 2 Terabytes.ii. For Linux 2.6 (when using 64 bit addressing on a
64 bit platform): 9 Petabytes (or the device limits)iii. For Linux 2.6 (on 32 bit platforms): 16Terabytes
(even with 64 bit addressing)
c) Maximum file size is same as filesystem size
Jonathan Wheeler
e-Science Centre, RAL
XFS Features (3)
Partitioned into Allocation Groups:
a) each AG manages its own free space and inodes
b) provides scalability and parallelism within the file system
c) limits the size of the structures needed to track this information
Jonathan Wheeler
e-Science Centre, RAL
XFS Features (3, continued)
d) allows many internal pointers to be 32-bits
e) AGs typically range in size from 0.5 to 4GB
f) files and directories are not limited to a single AG.
Jonathan Wheeler
e-Science Centre, RAL
XFS Features (4)
Sophisticated support utilities:a) fast mkfs (make a file system)b) dump and restore utilities for backupc) xfsrepair to fix corrupt filesystemd) xfs_fsr (XFS defragmenter)e) xfsdb (XFS debug)f) xfscheck (XFS check)g) xfs_growfs (enlarges filesystems online)
Jonathan Wheeler
e-Science Centre, RAL
Why you might want to use XFS
a) Stable, mature code base (oldest journaling filesystem available on Unix)
b) Very good performance for large reads or writes
c) Designed for large systemsd) Supports user and group quotase) Support for Access Control Lists and
Extended Attributes
Jonathan Wheeler
e-Science Centre, RAL
Extended Attributes
An extended attribute is a name/value pair associated with a file. Attributes can be attached to all types of inodes: regular files, directories, symbolic links, device nodes, and so forth. Attribute values can contain up to 64KB of arbitrary binary data.
See man attr for more details
Jonathan Wheeler
e-Science Centre, RAL
Extended attribute namespaces
XFS implements three attribute namespaces:a) user namespace available to all users,
protected by the normal file permissionsb) system namespace, accessible only to
privileged users and used for protected filesystem meta-data such as access control lists (ACLs) and hierarchical storage manager (HSM) file migration status
c) security namespace used by security modules (SELinux)
Jonathan Wheeler
e-Science Centre, RAL
Why you might not want to use XFS
a) No data journaling
b) tar, cpio do not understand Extended Attributes
Jonathan Wheeler
e-Science Centre, RAL
Resources
XFS/Linux homepage: http://oss.sgi.com/projects/xfs/ contains pointers to whitepapers, books, articles, etc.
Public mailing list: [email protected] with searchable list archive
SGI Bugzilla database to report any bugs in XFS for Linux
Jonathan Wheeler
e-Science Centre, RAL
Comparison tests - 1
Tests were performed:• on the same filesystem on the same
hardware (AMD Athlon 1666 Mhz, 500Mb)• using 3 different operating systems:
i. Redhat 7.3 (2.4.20-28.7)
ii. Scientific Linux version 3 (2.4.21-20.EL)
iii. Scientific Linux version 4
(2.6.9-22.0.1.EL and 2.6.9.-11.EL.XFS)
Jonathan Wheeler
e-Science Centre, RAL
Comparison tests - 2
Tests used different filesystem types:
• RH 7.3 – ext2, ext3 (xfs not available)
• SL 3 – ext2, ext3 (attempts to create xfs filesystem caused system crash requiring hardware reset)
• SL 4 – ext2, ext3, xfs (default), xfs (custom: mkfs.xfs –d agsize=4g –i version=1 –I size=512 <device>)
Jonathan Wheeler
e-Science Centre, RAL
Comparison tests - 3
Tests were performed:• using bonnie++ test program (see
http://www.coker.com.au/bonnie++)• using the same parameters for each test:
/usr/local/sbin/bonnie++ -d /data \ -s 4096M:1024k \-n 192:1048576:128:1024 -m csfnfs30 \-u root 1>> outputfile 2>&1
Jonathan Wheeler
e-Science Centre, RAL
Comparison tests - 4
12 tests in total, each reporting filesystem performance and CPU use
The first 6 tests measure I/O intensive work:
• Sequential output for character, blocked and rewrites
• Sequential input for character and blocked
• Random seeks
Jonathan Wheeler
e-Science Centre, RAL
Comparison tests - 6
The other 6 tests measure performance for many small files using:
• Sequential create, read and delete
• Random create, read and delete
Jonathan Wheeler
e-Science Centre, RAL
Comparsion of filesystems (Sequential output)
0
5000
10000
15000
20000
25000
30000
35000
73.ext2 73.ext3 sl3.ext2 sl3.ext3 sl4.ext2 sl4.ext3 sl4.xfs sl4.xfs.special
Filesystem type
Per c
hara
cter
(Kb/
sec)
Series1
Jonathan Wheeler
e-Science Centre, RAL
Comparison of filesystems (Sequential output)
0
5000
10000
15000
20000
25000
30000
35000
73.ext2 73.ext3 sl3.ext2 sl3.ext3 sl4.ext2 sl4.ext3 sl4.xfs sl4.xfs.special
Filesystem type
Per b
lock
(Kb/
sec)
Series1
Jonathan Wheeler
e-Science Centre, RAL
Comparison of filesystems (Sequential output)
0
5000
10000
15000
20000
25000
73.ext2 73.ext3 sl3.ext2 sl3.ext3 sl4.ext2 sl4.ext3 sl4.xfs sl4.xfs.special
Filesystem type
Rew
rite
(Kb/
sec)
Series1
Jonathan Wheeler
e-Science Centre, RAL
Comparsion of filesystems (Sequential input)
0
5000
10000
15000
20000
25000
30000
35000
73.ext2 73.ext3 sl3.ext2 sl3.ext3 sl4.ext2 sl4.ext3 sl4.xfs sl4.xfs.special
Filesystem type
Per c
hara
cter
(Kb/
sec)
Series1
Jonathan Wheeler
e-Science Centre, RAL
Comparison of filesystems (Sequential input)
80000
82000
84000
86000
88000
90000
92000
94000
96000
98000
100000
73.ext2 73.ext3 sl3.ext2 sl3.ext3 sl4.ext2 sl4.ext3 sl4.xfs sl4.xfs.special
Filesystem type
Per b
lock
(Kb/
sec)
Series1
Jonathan Wheeler
e-Science Centre, RAL
Compariosn of filesystems (Random seeks)
0
5
10
15
20
25
73.ext2 73.ext3 sl3.ext2 sl3.ext3 sl4.ext2 sl4.ext3 sl4.xfs sl4.xfs.special
Filesystem type
Num
ber/s
ec
Series1
Jonathan Wheeler
e-Science Centre, RAL
Comparison of filesystems (Sequential create)
0
5
10
15
20
25
30
35
40
45
73.ext2 73.ext3 sl3.ext2 sl3.ext3 sl4.ext2 sl4.ext3 sl4.xfs sl4.xfs.special
Filesystem type
Crea
te (N
um/s
ec)
Series1
Jonathan Wheeler
e-Science Centre, RAL
Comparison of filesystems (Sequential create)
0
20
40
60
80
100
120
140
160
180
73.ext2 73.ext3 sl3.ext2 sl3.ext3 sl4.ext2 sl4.ext3 sl4.xfs sl4.xfs.special
Filesystem type
Read
(num
ber/s
ec)
Series1
Jonathan Wheeler
e-Science Centre, RAL
Comparison of filesystems
0
50
100
150
200
250
300
350
400
73.ext2 73.ext3 sl3.ext2 sl3.ext3 sl4.ext2 sl4.ext3 sl4.xfs sl4.xfs.special
Filesystem type
Dele
te (n
umbe
r/sec
)
Series1
Jonathan Wheeler
e-Science Centre, RAL
Comparison of filesystems (Random create)
0
5
10
15
20
25
73.ext2 73.ext3 sl3.ext2 sl3.ext3 sl4.ext2 sl4.ext3 sl4.xfs sl4.xfs.special
Filesystem type
Crea
te (N
umbe
r/sec
)
Series1
Jonathan Wheeler
e-Science Centre, RAL
Comparison of filesystems (Random create)
0
5
10
15
20
25
73.ext2 73.ext3 sl3.ext2 sl3.ext3 sl4.ext2 sl4.ext3 sl4.xfs sl4.xfs.special
Filesystem type
Read
(Num
ber/s
ec)
Series1
Jonathan Wheeler
e-Science Centre, RAL
Comparison of filesystems (Random create)
0
5
10
15
20
25
30
35
40
45
50
73.ext2 73.ext3 sl3.ext2 sl3.ext3 sl4.ext2 sl4.ext3 sl4.xfs sl4.xfs.special
Filesystem type
Dele
te (N
umbe
r/sec
)
Series1