introduction to xfs hepsysman 5 th december 2005

32
Jonathan Wheeler E-Science Centre, RAL Introduction to XFS HEPSYSMAN 5 th December 2005

Upload: halle

Post on 15-Jan-2016

42 views

Category:

Documents


2 download

DESCRIPTION

Introduction to XFS HEPSYSMAN 5 th December 2005. What exactly is a filesystem ?. Organization: Large disk space to store small data items Use files and directories In UNIX everything is a stream of bytes: More complex file definition in other OS’s - PowerPoint PPT Presentation

TRANSCRIPT

Page 1: Introduction to XFS HEPSYSMAN 5 th  December 2005

Jonathan Wheeler

E-Science Centre, RAL

Introduction to XFS

HEPSYSMAN

5th December 2005

Page 2: Introduction to XFS HEPSYSMAN 5 th  December 2005

Jonathan Wheeler

e-Science Centre, RAL

What exactly is a filesystem ?

Organization:• Large disk space to store small data items• Use files and directories

In UNIX everything is a stream of bytes:• More complex file definition in other OS’s• Devices are also presented in the filesystem• Linux uses virtual filesystems, e.g. /proc

Page 3: Introduction to XFS HEPSYSMAN 5 th  December 2005

Jonathan Wheeler

e-Science Centre, RAL

A little bit of background

Three generation of UNIX filesystems:

a) v7 / sysv / coherent / minix

b) ffs / ext2

c) vxfs / ext3 / jfs / xfs / reiserfs

Page 4: Introduction to XFS HEPSYSMAN 5 th  December 2005

Jonathan Wheeler

e-Science Centre, RAL

Features of 3rd generation UNIX filesystems

a) Intent logging / journaling

b) Flexible metadata structures

c) Dynamic inode allocations

d) Extents

Page 5: Introduction to XFS HEPSYSMAN 5 th  December 2005

Jonathan Wheeler

e-Science Centre, RAL

History of XFS

Developed by Silicon Graphics (now SGI) for use on IRIX:

• 1993 - original design• 1994 - available for IRIX v 5.3• 2000 - released as Open Source• 2001/2002 – some Linux distributions with

native XFS support• 2003 – XFS support in 2.4 kernel

Page 6: Introduction to XFS HEPSYSMAN 5 th  December 2005

Jonathan Wheeler

e-Science Centre, RAL

XFS features (1)

Journaling:

• but only metadata updates, not data updates.

Fast Transactions:

• uses efficient B+ tree structures for fast searches and rapid space allocation

Page 7: Introduction to XFS HEPSYSMAN 5 th  December 2005

Jonathan Wheeler

e-Science Centre, RAL

XFS Features (2)

64-bit file system:a) 64bit variables for global counters and disk addresses

b) Large theoretical maximum filesystem sizei. For Linux 2.4: 2 Terabytes.ii. For Linux 2.6 (when using 64 bit addressing on a

64 bit platform): 9 Petabytes (or the device limits)iii. For Linux 2.6 (on 32 bit platforms): 16Terabytes

(even with 64 bit addressing)

c) Maximum file size is same as filesystem size

Page 8: Introduction to XFS HEPSYSMAN 5 th  December 2005

Jonathan Wheeler

e-Science Centre, RAL

XFS Features (3)

Partitioned into Allocation Groups:

a) each AG manages its own free space and inodes

b) provides scalability and parallelism within the file system

c) limits the size of the structures needed to track this information

Page 9: Introduction to XFS HEPSYSMAN 5 th  December 2005

Jonathan Wheeler

e-Science Centre, RAL

XFS Features (3, continued)

d) allows many internal pointers to be 32-bits

e) AGs typically range in size from 0.5 to 4GB

f) files and directories are not limited to a single AG.

Page 10: Introduction to XFS HEPSYSMAN 5 th  December 2005

Jonathan Wheeler

e-Science Centre, RAL

XFS Features (4)

Sophisticated support utilities:a) fast mkfs (make a file system)b) dump and restore utilities for backupc) xfsrepair to fix corrupt filesystemd) xfs_fsr (XFS defragmenter)e) xfsdb (XFS debug)f) xfscheck (XFS check)g) xfs_growfs (enlarges filesystems online)

Page 11: Introduction to XFS HEPSYSMAN 5 th  December 2005

Jonathan Wheeler

e-Science Centre, RAL

Why you might want to use XFS

a) Stable, mature code base (oldest journaling filesystem available on Unix)

b) Very good performance for large reads or writes

c) Designed for large systemsd) Supports user and group quotase) Support for Access Control Lists and

Extended Attributes

Page 12: Introduction to XFS HEPSYSMAN 5 th  December 2005

Jonathan Wheeler

e-Science Centre, RAL

Extended Attributes

An extended attribute is a name/value pair associated with a file. Attributes can be attached to all types of inodes: regular files, directories, symbolic links, device nodes, and so forth. Attribute values can contain up to 64KB of arbitrary binary data.

See man attr for more details

Page 13: Introduction to XFS HEPSYSMAN 5 th  December 2005

Jonathan Wheeler

e-Science Centre, RAL

Extended attribute namespaces

XFS implements three attribute namespaces:a) user namespace available to all users,

protected by the normal file permissionsb) system namespace, accessible only to

privileged users and used for protected filesystem meta-data such as access control lists (ACLs) and hierarchical storage manager (HSM) file migration status

c) security namespace used by security modules (SELinux)

Page 14: Introduction to XFS HEPSYSMAN 5 th  December 2005

Jonathan Wheeler

e-Science Centre, RAL

Why you might not want to use XFS

a) No data journaling

b) tar, cpio do not understand Extended Attributes

Page 15: Introduction to XFS HEPSYSMAN 5 th  December 2005

Jonathan Wheeler

e-Science Centre, RAL

Resources

XFS/Linux homepage: http://oss.sgi.com/projects/xfs/ contains pointers to whitepapers, books, articles, etc.

Public mailing list: [email protected] with searchable list archive

SGI Bugzilla database to report any bugs in XFS for Linux

Page 16: Introduction to XFS HEPSYSMAN 5 th  December 2005

Jonathan Wheeler

e-Science Centre, RAL

Comparison tests - 1

Tests were performed:• on the same filesystem on the same

hardware (AMD Athlon 1666 Mhz, 500Mb)• using 3 different operating systems:

i. Redhat 7.3 (2.4.20-28.7)

ii. Scientific Linux version 3 (2.4.21-20.EL)

iii. Scientific Linux version 4

(2.6.9-22.0.1.EL and 2.6.9.-11.EL.XFS)

Page 17: Introduction to XFS HEPSYSMAN 5 th  December 2005

Jonathan Wheeler

e-Science Centre, RAL

Comparison tests - 2

Tests used different filesystem types:

• RH 7.3 – ext2, ext3 (xfs not available)

• SL 3 – ext2, ext3 (attempts to create xfs filesystem caused system crash requiring hardware reset)

• SL 4 – ext2, ext3, xfs (default), xfs (custom: mkfs.xfs –d agsize=4g –i version=1 –I size=512 <device>)

Page 18: Introduction to XFS HEPSYSMAN 5 th  December 2005

Jonathan Wheeler

e-Science Centre, RAL

Comparison tests - 3

Tests were performed:• using bonnie++ test program (see

http://www.coker.com.au/bonnie++)• using the same parameters for each test:

/usr/local/sbin/bonnie++ -d /data \ -s 4096M:1024k \-n 192:1048576:128:1024 -m csfnfs30 \-u root 1>> outputfile 2>&1

Page 19: Introduction to XFS HEPSYSMAN 5 th  December 2005

Jonathan Wheeler

e-Science Centre, RAL

Comparison tests - 4

12 tests in total, each reporting filesystem performance and CPU use

The first 6 tests measure I/O intensive work:

• Sequential output for character, blocked and rewrites

• Sequential input for character and blocked

• Random seeks

Page 20: Introduction to XFS HEPSYSMAN 5 th  December 2005

Jonathan Wheeler

e-Science Centre, RAL

Comparison tests - 6

The other 6 tests measure performance for many small files using:

• Sequential create, read and delete

• Random create, read and delete

Page 21: Introduction to XFS HEPSYSMAN 5 th  December 2005

Jonathan Wheeler

e-Science Centre, RAL

Comparsion of filesystems (Sequential output)

0

5000

10000

15000

20000

25000

30000

35000

73.ext2 73.ext3 sl3.ext2 sl3.ext3 sl4.ext2 sl4.ext3 sl4.xfs sl4.xfs.special

Filesystem type

Per c

hara

cter

(Kb/

sec)

Series1

Page 22: Introduction to XFS HEPSYSMAN 5 th  December 2005

Jonathan Wheeler

e-Science Centre, RAL

Comparison of filesystems (Sequential output)

0

5000

10000

15000

20000

25000

30000

35000

73.ext2 73.ext3 sl3.ext2 sl3.ext3 sl4.ext2 sl4.ext3 sl4.xfs sl4.xfs.special

Filesystem type

Per b

lock

(Kb/

sec)

Series1

Page 23: Introduction to XFS HEPSYSMAN 5 th  December 2005

Jonathan Wheeler

e-Science Centre, RAL

Comparison of filesystems (Sequential output)

0

5000

10000

15000

20000

25000

73.ext2 73.ext3 sl3.ext2 sl3.ext3 sl4.ext2 sl4.ext3 sl4.xfs sl4.xfs.special

Filesystem type

Rew

rite

(Kb/

sec)

Series1

Page 24: Introduction to XFS HEPSYSMAN 5 th  December 2005

Jonathan Wheeler

e-Science Centre, RAL

Comparsion of filesystems (Sequential input)

0

5000

10000

15000

20000

25000

30000

35000

73.ext2 73.ext3 sl3.ext2 sl3.ext3 sl4.ext2 sl4.ext3 sl4.xfs sl4.xfs.special

Filesystem type

Per c

hara

cter

(Kb/

sec)

Series1

Page 25: Introduction to XFS HEPSYSMAN 5 th  December 2005

Jonathan Wheeler

e-Science Centre, RAL

Comparison of filesystems (Sequential input)

80000

82000

84000

86000

88000

90000

92000

94000

96000

98000

100000

73.ext2 73.ext3 sl3.ext2 sl3.ext3 sl4.ext2 sl4.ext3 sl4.xfs sl4.xfs.special

Filesystem type

Per b

lock

(Kb/

sec)

Series1

Page 26: Introduction to XFS HEPSYSMAN 5 th  December 2005

Jonathan Wheeler

e-Science Centre, RAL

Compariosn of filesystems (Random seeks)

0

5

10

15

20

25

73.ext2 73.ext3 sl3.ext2 sl3.ext3 sl4.ext2 sl4.ext3 sl4.xfs sl4.xfs.special

Filesystem type

Num

ber/s

ec

Series1

Page 27: Introduction to XFS HEPSYSMAN 5 th  December 2005

Jonathan Wheeler

e-Science Centre, RAL

Comparison of filesystems (Sequential create)

0

5

10

15

20

25

30

35

40

45

73.ext2 73.ext3 sl3.ext2 sl3.ext3 sl4.ext2 sl4.ext3 sl4.xfs sl4.xfs.special

Filesystem type

Crea

te (N

um/s

ec)

Series1

Page 28: Introduction to XFS HEPSYSMAN 5 th  December 2005

Jonathan Wheeler

e-Science Centre, RAL

Comparison of filesystems (Sequential create)

0

20

40

60

80

100

120

140

160

180

73.ext2 73.ext3 sl3.ext2 sl3.ext3 sl4.ext2 sl4.ext3 sl4.xfs sl4.xfs.special

Filesystem type

Read

(num

ber/s

ec)

Series1

Page 29: Introduction to XFS HEPSYSMAN 5 th  December 2005

Jonathan Wheeler

e-Science Centre, RAL

Comparison of filesystems

0

50

100

150

200

250

300

350

400

73.ext2 73.ext3 sl3.ext2 sl3.ext3 sl4.ext2 sl4.ext3 sl4.xfs sl4.xfs.special

Filesystem type

Dele

te (n

umbe

r/sec

)

Series1

Page 30: Introduction to XFS HEPSYSMAN 5 th  December 2005

Jonathan Wheeler

e-Science Centre, RAL

Comparison of filesystems (Random create)

0

5

10

15

20

25

73.ext2 73.ext3 sl3.ext2 sl3.ext3 sl4.ext2 sl4.ext3 sl4.xfs sl4.xfs.special

Filesystem type

Crea

te (N

umbe

r/sec

)

Series1

Page 31: Introduction to XFS HEPSYSMAN 5 th  December 2005

Jonathan Wheeler

e-Science Centre, RAL

Comparison of filesystems (Random create)

0

5

10

15

20

25

73.ext2 73.ext3 sl3.ext2 sl3.ext3 sl4.ext2 sl4.ext3 sl4.xfs sl4.xfs.special

Filesystem type

Read

(Num

ber/s

ec)

Series1

Page 32: Introduction to XFS HEPSYSMAN 5 th  December 2005

Jonathan Wheeler

e-Science Centre, RAL

Comparison of filesystems (Random create)

0

5

10

15

20

25

30

35

40

45

50

73.ext2 73.ext3 sl3.ext2 sl3.ext3 sl4.ext2 sl4.ext3 sl4.xfs sl4.xfs.special

Filesystem type

Dele

te (N

umbe

r/sec

)

Series1