File system 3Nezer J. Zaidenberg

Agenda

• Minhala

• Final presentation of Linux FS code

• My personal summary of the semester

Siurhazara

• 12/2/09 Dach 14:00-16:00

• We shall review all questions from exam sample and oral exams

• Attendance is not required.

• But if you come bring food and soft drinks

Userland reading

APUE e2

• Files and dirs - Chapter 3 (and a little of 4)

• Process - Chapter 7 + 8 + relevant subsections of 9 + 10 (signals) +13 (Daemon)

• Threads and sync (11+12)

• I/O multiplexing and select (beginning of chapter 14)

• Sockets (Internet and UDS) –Chapter 16,17.3 or Beej

APUE e1• Chapter 3

• Chapter 7 (up to subsection 10)

• Chapter 8

• Chapter 9 (relevant subsections)

• Chapter 10

• Chapter 11 (select)

• Chapter 13

• Also Sockets + Threads

• Relevant subsections from chapters 4,5

Kernel reading for the exam

• Understanding the linux kernel 3rd edition chapters • Chapter 12. The Virtual Filesystem• Chapter 14. Block Device Drivers• Chapter 15. The Page Cache• Chapter 16. Accessing Files• Chapter 18. The Ext2 and Ext3 Filesystems

• UNIX filesystems• Chapter 10• Chapter 12• Chapter 14

• As Kernel versions progress and API changes (and my desire to keep the course on bleeding edge.) not all API’s (such as XX_iget()) can be found in books. Check my slides.

Last train to Clarksville

• I’ve sent the questions of the Oral exam to Amir he will upload Sunday

• IF YOU HAVE SUBMITTED AND NOT TESTED TODAY IS YOUR ABSOLUTLY POSITIVELY DEFINTELY REALLY LAST CHANCE

• Please approach me at 17:00 or between 15:00 and 16:00

• If you have not taken oral exam your grade in the ex. Will be ZERO even if you have submitted!

Please return DVD’s

• Even though I created 10 DVD’s for about 30 teams each needed for about 10 minutes there are 2 and a lot of time there were 0 (ZERO).

• PLEASE – If you have taken a DVD from the envelope return it!!!!

And back to diggingLinks and mmap

What we should know• Where to find kernel sources

• How to mount file systems

• Basic recognition with the source to create, access and remove files and directories.

• How to read and write from block device

• I’ve also given you a short manual on how to write a simple file system in memory.

• We have only two gaps

• Building links

• Building directories

• Following request we will also breeze through mkfs

mkfs

• What does mkfs supposed to do• Mkfs just MAKES a blank file system on a device

(In windows we would call it FORMAT a disk)• Usually that means initializing the super block

(writing magic number, writing blocks as empty etc.) and writing it to disk. Initializing the root directory and writing it to disk. Initializing other structs (bitmaps) and writing them to disk. Initializing other directories (lost+found) and writing them to disk.

• Just to be clear mkfs is a userland program it should not be more then few hundreds lines of code.

Book mkfs 1/4

main(intargc, char **argv)

{

… (variable definition and sanity check skipped)

devfd = open(argv[1], O_WRONLY); // line 31

error = lseek(devfd, (off_t)(nsectors * 512), SEEK_SET); //make sure device is large enough // line 36

lseek(devfd, 0, SEEK_SET); // line 42

sb.s_magic = UX_MAGIC;

sb.s_mod = UX_FSCLEAN;

sb.s_nifree = UX_MAXFILES - 4;

sb.s_nbfree = UX_MAXBLOCKS - 2;

sb.s_inode[0] = UX_INODE_INUSE;

Book mkfs 2/4sb.s_inode[1] = UX_INODE_INUSE;

sb.s_inode[2] = UX_INODE_INUSE;

sb.s_inode[3] = UX_INODE_INUSE;

for (i = 4 ; i< UX_MAXFILES ; i++) {

sb.s_inode[i] = UX_INODE_FREE; }

sb.s_block[0] = UX_BLOCK_INUSE; sb.s_block[1] = UX_BLOCK_INUSE;

for (i = 2 ; i< UX_MAXBLOCKS ; i++) {

sb.s_block[i] = UX_BLOCK_FREE; }

write(devfd, (char *)&sb, sizeof(structux_superblock)); // line 89

Book mkfs 3/4time(&tm);

memset((void *)&inode, 0, sizeof(structux_inode));

inode.i_mode = S_IFDIR | 0755;

inode.i_nlink = 3; /* ".", ".." and "lost+found" */

inode.i_atime = tm; inode.i_mtime = tm; inode.i_ctime = tm;

inode.i_uid = 0; inode.i_gid = 0;

inode.i_size = UX_BSIZE;

inode.i_blocks = 1;

inode.i_addr[0] = UX_FIRST_DATA_BLOCK;

lseek(devfd, UX_INODE_BLOCK * UX_BSIZE + 1024, SEEK_SET);

write(devfd, (char *)&inode, sizeof(structux_superblock));

// the process repeats itself

Book mkfs 4/4lseek(devfd, UX_FIRST_DATA_BLOCK * UX_BSIZE, SEEK_SET);

memset((void *)&block, 0, UX_BSIZE);

write(devfd, block, UX_BSIZE);

lseek(devfd, UX_FIRST_DATA_BLOCK * UX_BSIZE, SEEK_SET);

dir.d_ino = 2;

strcpy(dir.d_name, ".");

write(devfd, (char *)&dir, sizeof(structux_dirent));

dir.d_ino = 2;

strcpy(dir.d_name, "..");

write(devfd, (char *)&dir, sizeof(structux_dirent));

dir.d_ino = 3;

strcpy(dir.d_name, "lost+found");

write(devfd, (char *)&dir, sizeof(structux_dirent));

mmap

• Mmapbasicly is read and write but works with pages

• A new operation struct is defined to deal with pages

• The reading and writing of pages is just as reading and writing of blocks and implemented in a similar way

• Lets take a look at ext2

Fs/ext2/inode.c

791 const structaddress_space_operations ext2_aops = {

792 .readpage = ext2_readpage,

793 .readpages = ext2_readpages,

794 .writepage = ext2_writepage,

795 .sync_page = block_sync_page,

796 .write_begin = ext2_write_begin,

797 .write_end = generic_write_end,

798 .bmap = ext2_bmap,

799 .direct_IO = ext2_direct_IO,

800 .writepages = ext2_writepages,

801 .migratepage = buffer_migrate_page,

802 .is_partially_uptodate = block_is_partially_uptodate,

803 };

The address space struct

1269 if (S_ISREG(inode->i_mode)) {

1270 inode->i_op = &ext2_file_inode_operations;

1278 inode->i_mapping->a_ops = &ext2_aops;

1279 inode->i_fop = &ext2_file_operations;

1280 }

1281 } else if (S_ISDIR(inode->i_mode)) {

1282 inode->i_op = &ext2_dir_inode_operations;

1283 inode->i_fop = &ext2_dir_operations;

1287 inode->i_mapping->a_ops = &ext2_aops;

• The code is from ext2/fs/inode.c

• I removed some if-clauses

• Basicly we set the mmap function pointers just like we set the inode operators

Fs/ext2/inode.c 715 static int ext2_writepage(struct page *page,

structwriteback_control *wbc)

716 {

717 return block_write_full_page(page, ext2_get_block, wbc);

718 }

719

720 static int ext2_readpage(struct file *file, struct page *page)

721 {

722 return mpage_readpage(page, ext2_get_block);

723 }

Fs/ext2/inode.c

725 static int

726 ext2_readpages(struct file *file, struct address_space *mapping,

727 struct list_head *pages, unsigned nr_pages)

728 {

729 return mpage_readpages(mapping, pages, nr_pages, ext2_get_block);

730 }

translation

• Linux functions are called to get the memory page. Those functions gets a pointer to a function to a ext2 function that ACTUALLY does the block reading from disk. Once the block is read it is placed in memory page(s) by the kernel.

• The Kernel function that we call mpage_readpage(s) is found under lines 370-420 of fs/mpage.cblock_write_full_page is at 2859-2894 of fs/buffer.c

• I will not dwell into those function because of time limitation. (Those functions call other functions and are beyond out time limits. However, they are contained inside the files and are not really required.)

• Take note that readpage is the same function we would call when we read(2)

Links

• HARD LINKS• Just another pointers (in a directory) to the

same file • (no new inode)

• SOFT (Symbolic) Link (bonus in ex.)• Block contains the file we point kernel does

the rest

• Lets dig into ext2 (hard links first)

The struct

360 const structinode_operations ext2_dir_inode_operations = {

361 .create = ext2_create,

362 .lookup = ext2_lookup,

363 .link = ext2_link,

364 .unlink = ext2_unlink,

365 .symlink = ext2_symlink,

366 .mkdir = ext2_mkdir,

367 .rmdir = ext2_rmdir,

368 .mknod = ext2_mknod,

369 .rename = ext2_rename,

376 .setattr = ext2_setattr,

377 .permission = ext2_permission,

378 };

translation

• The struct can be found at fs/ext2/namei.c

• The relevant functons are called when we link(2) and unlink(2)

Fs/ext2/namei.c177 static int ext2_link (structdentry * old_dentry, structinode * dir, structdentry *dentry)

179 {

180 structinode *inode = old_dentry->d_inode;

182 if (inode->i_nlink>= EXT2_LINK_MAX)

183 return -EMLINK;

185 inode->i_ctime = CURRENT_TIME_SEC;

186 inode_inc_link_count(inode);

187 atomic_inc(&inode->i_count);

189 return ext2_add_nondir(dentry, inode);

190 }

Fs/ext2/namei.c 39 static inline int ext2_add_nondir(struct dentry *dentry, structinode *inode)

40 {

41 int err = ext2_add_link(dentry, inode);

42 if (!err) {

43 d_instantiate(dentry, inode);

44 return 0;

45 }

46 inode_dec_link_count(inode);

47 iput(inode);

48 return err;

49 }

Fs/ext2/dir.c473 int ext2_add_link (structdentry *dentry, structinode *inode)

474 {

//find page

545 if (de->inode) {

546 ext2_dirent *de1 = (ext2_dirent *) ((char *) de + name_len);

547 de1->rec_len = ext2_rec_len_to_disk(rec_len - name_len);

548 de->rec_len = ext2_rec_len_to_disk(name_len);

549 de = de1;

550 }

558 mark_inode_dirty(dir);

Using kernel APId_instantiate (9)

Name

d_instantiate- fill in inode information for a dentry

Synopsis

void d_instantiate(structdentry * entry, structinode * inode);

Arguments

Entry dentryto complete

Inodeinodeto attach to this dentry

Description

Fill in inode information in the entry.

This turns negative dentries into productive full members of society.

NOTE! This assumes that the inode count has been incremented (or otherwise set) by the caller to indicate that it is now in use by the dcach

Symbolic links

• Briefly… they are created using the symlink(2) syscall

• From fs/ext2/inode.c

1289 if (ext2_inode_is_fast_symlink(inode))

1290 inode->i_op = &ext2_fast_symlink_inode_operations;

The symlink functions

360 const structinode_operations ext2_dir_inode_operations = {

361 .create = ext2_create,

362 .lookup = ext2_lookup,

363 .link = ext2_link,

364 .unlink = ext2_unlink,

365 .symlink = ext2_symlink,

366 .mkdir = ext2_mkdir,

367 .rmdir = ext2_rmdir,

368 .mknod = ext2_mknod,

.rename = ext2_rename,

376 .setattr = ext2_setattr,

377 .permission = ext2_permission,

378 };

fs/ext2/namei.c133 static int ext2_symlink (structinode * dir, structdentry * dentry,

134 const char * symname)

135 {

136 structsuper_block * sb = dir->i_sb;

137 int err = -ENAMETOOLONG;

138 unsigned l = strlen(symname)+1;

139 structinode * inode;

141 if (l>sb->s_blocksize)

142 goto out;

144 inode = ext2_new_inode (dir, S_IFLNK | S_IRWXUGO);

145 err = PTR_ERR(inode);

146 if (IS_ERR(inode))

fs/ext2/namei.c147 goto out;

149 if (l>sizeof (EXT2_I(inode)->i_data)) {

150 /* slow symlink */

151 inode->i_op = &ext2_symlink_inode_operations;

155 inode->i_mapping->a_ops = &ext2_aops;

156 err = page_symlink(inode, symname, l);

157 if (err)

158 gotoout_fail;

165 mark_inode_dirty(inode);

167 err = ext2_add_nondir(dentry, inode);

168 out:

169 return err;

General home work QA

• Where can we find the kernel?• www.kernel.org

• Why is there no WORKING solution supplied by you?• There are at least 10 solutions you can find in the kernel – minix,

ext2/3/4, ufs, reiser_fs, xfs, sysv, jfs, hpfs and the entire point was to get you to dig there

• Since when are homework supplied with solutions anyway.

• Where can we find mkfs.minix• Mkfs.minix is NOT part of the linux kernel. It is user application that

is part of linux-utils• I found it under linux-utils version 2.12q (downloaded and untarred

sources, ./configure and make. Compilation was broken but after I have built mkfs.minix (didn’t bothered to correct the errors not needed)

• I understand newer versions are available but I never tried them. I understand some students got them to work as well

We are still at loss how do we start working on this

ex.• Copy minix tree to your playground directory, build mkfs.minix (use man --path

“path of mkfs.minix.8” which comes with linuxutils if you want to see man)

• Edit the source for minixmakefile so that it will call linuxmakefile recursively

• For each function add “printk” that print the function name. (and __FILE__ and__LINE__)

• Format the floppy to minix and mount it. Check syslog

• Start running operations. Check syslog. Check which functions are called immediately, which functions are called after a while (writing buffer to disks) – check timestamps

• Either take minix, remove the dead cod eand stuff you don’t need, and learn it or take book example and start fixing things till its working

• The experience should be entertaining

• Maybe it will be best if you access the file system via C code that calls system calls and not via shell commands who may do other things. (that may confuse you)

General test FAQ• There are 36 quiz questions in the test (1 answer in 4)

• 3pts correct answer and no deduction on incorrect answer (total 108)

• Not an open book exam. (we added the Synopsis of all functions you may need. )

• No WINDOWS

• Most of the exam is based on homework (Including ex.3) but there are theoretical questions on the mmap

• The exam has about 50% of its question based on the questions we asked in the homework (modified to fit quiz format)

• To study – learn your code and follow the presentations on file system, sync, virtual memory, GRID.

• We put some questions regarding using Linux for software development. (i.e. regarding gdb, gcc, man, make, strace etc.)

• Don’t be surprised if you finish the exam in 15m….

emphasis on

• Using system calls

• Using UNIX

• Multitasking (processes, threads)

• Communications

• I/O multiplexing (select)

• File I/O (open/read/write/mmap etc.)

• File systems and kernel

Just some personal summary

My successful objectives for this course

• Provide the class with Solid understanding of userland interface to the OS• Achieved

• Provide the class with means to add and review code in the Linux kernel• Partially achieved (we have studied the file system interface)

• Make the students a better programmers, more proficient debuggers and more efficient and solving their own problems…• Achieved for most students (making some students better

programmers was easier then I suspected)

• Provide the students with confidence in their abilities to code new interfaces studying from user manual and existing code • Achieved

• Survive this class without being burned to the stake ;-)• Achieved (but then I am getting married so maybe being burned to

the stake is not that bad…)

My failed objectives for this course

I’ve hoped to get to do a lot more kernel stuff

Replace scheduler

Write char device driver

Write kernel module that communicates with user land programs

Maybe next year I will have more luck

I’ve hoped to get more guest lectures from the industry and failed. (most companies no longer need to make a name for themselves)

I’ve hoped to inflict most students with the Linux germ as most efficient programming environment

Things I’ve done good

• Adjust the course to the level of the class

• Teach most of the class to work on strict timeline

• Receive working exercises with no resubmissions

• Maintain interest

• Gain respect

Things that I haven’t

• Assist the students to much, possibly to a level were self-learning was damaged

• There were errors in some presentations which will be removed next time I teach the course.• Daemon code (I’ve removed error checking to fit things on

one slide and ended with if (pid = fork()<0))• Kernel 2.4 build method on some slides. (used to work also

on 2.6….not anymore)• Changed interface to file system on 2.6 (compared to

book)• Hard link == +1 Inode. (This is true in some

implementations were a circular list of inodes is kept but not on Linux)

• Dragged into too many argument during lectures with the class

If you have enjoyed the course and want to do

some hacking• There are plenty of OS related projects out there you can volunteer to contribute code• OSs : Linux, FreeBSD (BSD on intel with great performance), Darwin (OSX open

source), NetBSD (BSD for max portability), OpenBSD (BSD for max security)• Servers : Apache web server and many others• Tools : gcc, vi, emacs• Games : QFG2/KQ1 VGA remake, Ur Quan masters, GNU Chess, FreeCiv etc.• Libraries : ACE, nspr, ICE and others• Documentation : There are many parts that would benefit from good documentation

or manual• And you can start your own

• There are plenty others thing you can do (install Linux (website do constant surveys)), use open standards were you work, contribute helper projects to the community• In one of my companies we used DB which at the time didn’t have Linux GUI.

Developers that worked for me developed a Linux GUI that managed the DB. This work was donated to the Linux community

• Similarly one of my employees has donated Object Pascal syntax enlightment to vi (to use vi inside Borland delphi)

• Another one has donated Kernel netlink sockets (type of sockets that allow user and kernel space to communicate frameworks to ACE

• None of the above damaged our company IP and in fact when it was picked up and updated and tested by the community it helped us

Does it pay?Salary market survey

job student grad 2-3 3-5 5+ mng

C++/.NET 55-70 מתכנת 13-16

18-20 20-23 24-27 25-28

Java 55-70a 13-16 18-20 21-24 24-27 25-28 מתכנת

)QA( בדיקות 30-50 11-13 13-15 15-17 17-20 18-22

תצורה 18-21 15-17 12-14 9-11 11-13 בקרת

Drivers/Kernel מפתח 23-25 25-27 27-29 27-30

טכני 16-20 12-15 10-11 כתב

Real Time 60-75 14-16 19-21 22-25 25-28 26-28 מהנדס

28-31 26-28 23-25 19-22 15-17 ואלגוריתמים DSP מהנדס

תמיכה EAF 16-18 19-21 20-23 22-25 מהנדס

מערכת 28-31 25-28 23-26 מהנדס

Pre/Post Sale מהנדס 18-21 22-25 25-26

מנהלפרויקט 18-20 22-24 25-27

מנהלמוצר 20-22 23-25 26-28

• Job info (market survey Dec 2008)

0-2 3-5 5-10 manager

Java 16-21 18-23 23-28 26-29 מהנדס

Web מהנדס15-10

15-18 16-20 18-22

C/C 16-21 21-25 22-27 25-30++ מהנדס

Dot Net 12-17 מהנדס 18-23 23-28 25-30

RT Embedded 18-22 20-24 24-29 29-33 מהנדס

מהנדסאבטחתמידע 12-15 15-21 21-30 30-32

Linux Kernel 17-21 19-23 25-32 32-35 מהנדס

מהנדסמערכת 20-23 23-26 26-29 29-34

מנהלפרויקטבתוכנה 15-20 20-25 25-32 27-35

• CP&S (market survey October 2008)

How to reach me

• nzaidenberg@mac.com

• Students that have submitted all 3 exercises and want a reference from me should email me.

• I’ll be glad to assist in most UNIX/Linux real world scenarios you may encounter.

Good luck on the test

And remember… each time you use .Net a penguin dies.