Intro to DFS assign-ment

Announcements• DFS (PA4) Part 1 will be out over the weekend• Part 1 Due 10/30 11:59 pm • Part 2 12/7 11:59 pm• Part 3 12/14 11:59 pm• Part 4 (optional extra credit) 12/23 11:59pm

You build a DFS!1. Lock server, at-most-once RPC semantics2. Implement extent server; create/lookup/readdir FUSE ops3. Implement read/write/open/setattr4. Implement mkdir/unlink, integrate locks! (op-tional)

Programming Assignment: YFS• Yet-another File System. :)• Simpler version of what we just talked about: only

one extent server (you don’t have to implement Petal; single lock server)

Part 1: lock server• Implementing the lock server• Provide mutual exclusion to ‘lock resources’•Using pthread mutexes, condition variables!

• Implementing at-most-once RPC semantics

Providing mutualexclusion over the net• In the assignment: lock is a 64-bit number•Client might:• acquire(lock_a); do work; release(lock_a);•You must create the lock if it doesn’t yet exist on the server

Example tester% ./lock_tester 3772 simple lock client acquire a release a acquire a release a acquire a acquire b release b release a test2: client 0 acquire a release a test2: client 2 acquire a release a ...

What’s provided• Simple RPC framework, skeleton for lockserver code•What it does: handles setting up sockets, mar-shalling/unmarshalling basic data types, sending over TCP (whew)•What it doesn’t: does not keep track of RPC re-quest state; duplicate RPCs are invoked twice! :(

Example RPC call

Your job for lock server

Lossy• RPC_LOSSY: drops, duplicates, delays.• Run lock_tester with RPC_LOSSY=0

• Now try running it with RPC_LOSSY=5• Simulates network failure• it should hang or error. FAIL!

Why?• No at-most-once RPC semantics• implemented...yet.

• If reply dropped, duplicate sent: acquire(a), acquire(a)..............................

Strawman•Remember every RPC call•Client sends a unique RPC identifier

•Remember every RPC reply•To avoid invoking the actual function

•What’s the problem with this ap-proach?

Sliding window RPCs• Assume the following sent by client:marshall m1 << clt nonce // client id << svr nonce // server id<< proc // procedure number (bind, acquire, etc).<< myxid // unique request id for this RPC<< xid_rep_window.front() // Last out-of-order RPC reply received<< req.str(); // Data• But we need some additional info from the client

At the end% export RPC_LOSSY=5% ./lock_server 3772 &% ./lock_tester 3772...all tests passed% ./rpctestsimple_tests...all tests passed

Helper functions• Method_thread

Code reading

Part 2• Extent server• FUSE!• Semantics of filesystem calls

Programming Assignment: YFS• Part 1: lock server• Part 2: extent server

The extent server•Don’t have to deal with storing data on disk: you will store contents in memory•map<inode, std::string>

•All yfs_client operations synchronize with same extent server

Getattr (given)

Your job• Extend extent_server to support• put(extentid, string, ...)• get(extentid, string&)• remove(extentid, ...)• See extent_protocol.h and extent_smain.cc• Later you will likely add more!• Must properly deal with ctime/atime/mtime

Data formats at extent server•Directories maintain a mapping of filenames to inode numbers• if root (inode 1) has two files “file1”, “file2”: get(1) might return• file1:3983293923• file2:3384927553

• You can choose how you store this information as a string

Metadata time mgmt• atime: access time• Updated whenever file contents accessed• Set to 0 on file creation

• mtime: modification time• Updated when file contents modified

• ctime: change time• Updated whenever metadata modified

FUSE• http://fuse.sourceforge.net/

Mapping FUSE functions

Example• Fuse.cc• void• fuseserver_getattr(

CREATE/MKNOD•Generate inode number (rand())• This is called on files: make sure inode has MSB set to 1• Directories have MSB set to 0

• Create the file at the extent server• Add the name to inode mapping to parent info at extent server

Lookup•Given: filename, parent inode• Look through parent directory list• Find inode that maps from filename• Getattr on that inode

• Fill in return structure (fuse_entry_param)

Readdir• Given: inode number for a directory• Get filename:inode mapping for that dir• For each filename:inode pair call dirbuf_add(...) function

provided

• You will be able to create empty files in the root direc-tory and do an ls on the root• You’re not yet storing files...

Testing

• ./start.sh• ./test-lab-2.pl• ./stop.sh• test-lab-2.pl creates a bunch of files, tests whether the files the script created are there

Parts 3 and 4• Part 3: You implement open, read, write, setattr• Part 4: You implement mkdir, unlink, and locking