Skip to content

Commit 3130a2e

Browse files
remzi-arpacidusseauremzi-arpacidusseau
committed
next cut
1 parent a688f0e commit 3130a2e

File tree

1 file changed

+161
-1
lines changed

1 file changed

+161
-1
lines changed

‎filesystems-distributed/README.md‎

Lines changed: 161 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,165 @@
11

22
# Distributed File System
33

4-
In this project,
4+
In this assignment, you will be developing a working *distributed file
5+
server.* We provide you with only the bare minimal UDP communication
6+
code; you have to build the rest.
7+
8+
## A Basic File Server
9+
10+
Your file server is built as a stand-alone UDP-based server. It should wait
11+
for a message and then process the message as need be, replying to the given
12+
client.
13+
14+
Your file server will store all of its data in an on-disk file which will
15+
be referred to as the *file system image*. This image contains the on-disk
16+
representation of your data structures; you should use these system calls
17+
to access it: `open(), read(), write(), lseek(), close(), fsync().`
18+
19+
To access the file server, you will be building a client library. The
20+
interface that the library supports is defined in XXX MFS.H XXX. The
21+
library should be called `libmfs.so`, and any programs that wish to access
22+
your file server will link with it and call its various routines.
23+
24+
## On-Disk File System: A Log-Structured File System
25+
26+
Your on-disk file system structures should roughly follow that of the
27+
log-structured file system discussed in class. On-disk, the first structure
28+
should be a singular checkpoint region. The checkpoint region should contain a
29+
disk pointer to the current end of the log; it should also contain pointers
30+
to pieces of the inode map (assume there are a maximum of 4096 inodes; assume
31+
each piece of the inode map has 16 entries).
32+
33+
Other than the checkpoint region, your on-disk image just consists of an
34+
ever-growing log (i.e., we won't be implementing cleaning). Thus, whenever you
35+
write to the disk, you'll just write all file system updates to the end of the
36+
log, and then update the checkpoint region as need be. For example, if you are
37+
adding a new block to a file, you would write the data block, new version of
38+
the inode, and a new piece of the inode map to the end of the log; when this
39+
write completes, you should update the checkpoint region with the requisite
40+
new values.
41+
42+
The inode map is just an array, indexed by inode number. Each entry is a
43+
simple 4-byte integer, which is just the disk address of the location of the
44+
inode in question.
45+
46+
Each inode should be simple: a size field (the number of the last byte in
47+
the file), a type field (regular or directory), and 14 direct pointers; thus,
48+
the maximum file size is 14 times the 4KB block size, or 56 KB.
49+
50+
One other structure you'll have to manage on disk are directories. Each
51+
directory has an inode, and points to one or more data blocks that contain
52+
directory entries. Each directory entry should be simple, and consist of 32
53+
bytes: a name and an inode number pair. The name should be a fixed-length
54+
field of size 28 bytes; the inode number is just an integer (4 bytes). When a
55+
directory is created, it should contain two entries: the name ., which
56+
refers to this new directory's inode number, and .., which refers to the
57+
parent directory's inode number. For directory entries that are not yet in use
58+
(in an allocated 4-KB directory block), the inode number should be set to
59+
-1. This way, utilities can scan through the entries to check if they are
60+
valid.
61+
62+
When your server is started, it is passed the name of the file system image
63+
file. If this file does not exist, the file server should create it, and
64+
initialize it properly, and force it to disk. Such initialization includes
65+
creating the checkpoint region, the initial inode map, and creating a single
66+
root directory with proper . and .. entries. The root inode number should be 0.
67+
68+
When booting off of an existing image, your server should read in the
69+
checkpoint region (and keep an in-memory version of it), as well as the entire
70+
inode map and keep it in-memory too.
71+
72+
73+
## Client library
74+
75+
The client library should export the following interfaces:
76+
77+
- int MFS_Init(char *hostname, int port): MFS_Init() takes a host name
78+
and port number and uses those to find the server exporting the file system.
79+
- int MFS_Lookup(int pinum, char *name): MFS_Lookup() takes the parent
80+
inode number (which should be the inode number of a directory) and looks up
81+
the entry `name` in it. The inode number of `name` is returned. Success:
82+
return inode number of name; failure: return -1. Failure modes: invalid pinum,
83+
name does not exist in pinum.
84+
- int MFS_Stat(int inum, MFS_Stat_t *m): MFS_Stat() returns some
85+
information about the file specified by inum. Upon success, return 0,
86+
otherwise -1. The exact info returned is defined by MFS_Stat_t. Failure modes:
87+
inum does not exist.
88+
- int MFS_Write(int inum, char *buffer, int block): MFS_Write() writes a
89+
block of size 4096 bytes at the block offset specified by `block`. Returns 0
90+
on success, -1 on failure. Failure modes: invalid inum, invalid block, not a
91+
regular file (because you can't write to directories).
92+
- int MFS_Read(int inum, char *buffer, int block): MFS_Read() reads
93+
a block specified by `block` into the buffer from file specified by
94+
`inum`. The routine should work for either a file or directory;
95+
directories should return data in the format specified by
96+
MFS_DirEnt_t. Success: 0, failure: -1. Failure modes: invalid inum,
97+
invalid block.
98+
- int MFS_Creat(int pinum, int type, char *name): MFS_Creat() makes a
99+
file (`type == MFS_REGULAR_FILE`) or directory (`type == MFS_DIRECTORY`)
100+
in the parent directory specified by *pinum* of name *name*. Returns 0 on
101+
success, -1 on failure. Failure modes: pinum does not exist, or name is too
102+
long. If `name` already exists, return success (think about why).
103+
- int MFS_Unlink(int pinum, char *name): MFS_Unlink() removes the file or
104+
directory `name` from the directory specified by `pinum`. 0 on success, -1
105+
on failure. Failure modes: pinum does not exist, directory is NOT empty. Note
106+
that the name not existing is NOT a failure by our definition (think about why
107+
this might be).
108+
- int MFS_Shutdown(): MFS_Shutdown() just tells the server to force all
109+
of its data structures to disk and shutdown by calling exit(0). This interface
110+
will mostly be used for testing purposes.
111+
112+
113+
## Server Idempotency
114+
115+
The key behavior implemented by the server is *idempotency*.
116+
Specifically, on any change to the file system state (such as a
117+
MFS_Write, MFS_Creat, or MFS_Unlink), all the dirtied buffers in the
118+
server are committed to the disk. The server can achieved this end by
119+
calling `fsync()` on the file system image. Thus, before returning a
120+
success code, the file system should always `fsync()` the image.
121+
122+
Now you might be wondering: why do this? Simple: if the server crashes, the
123+
client can simply timeout and retry the operation and know that it is OK to do
124+
so. Read this chapter on NFS for details.
125+
126+
127+
Now you might be wondering: how do I implement a timeout? Simple, with the
128+
`select()` interface. The `select()` calls allows you to wait for a reply
129+
on a certain socket descriptor (or more than one, though that is not needed
130+
here). You can even specify a timeout so that the client does not block
131+
forever waiting for data to be returned from the server. By doing so, you can
132+
wait for a reply for a certain amount of time, and if nothing is returned, try
133+
the operation again until it is successful.
134+
135+
## Program Specifications
136+
137+
Your server program must be invoked exactly as follows:
138+
139+
prompt> server [portnum] [file-system-image]
140+
141+
The command line arguments to your file server are to be interpreted as follows.
142+
143+
- portnum: the port number that the file server should listen on.
144+
- file-system-image: a file that contains the file system image.
145+
146+
If the file system image does not exist, you should create it and properly
147+
initialize it to include an empty root directory.
148+
149+
Your client library should be called libmfs.so. It should implement
150+
the interface as specified by `mfs.h`, and in particular deal with
151+
the case where the server does not reply in a timely fashion; the way
152+
it deals with that is simply by retrying the operation, after a
153+
timeout of some kind (default: five second timeout).
154+
155+
## Some Helper Code
156+
157+
To get you going, we have written some simple UDP code that can send a
158+
message and then receive a reply from a client to a server. It can be found in
159+
the XXX.
160+
161+
How to make a shared library: [here](https://tldp.org/HOWTO/Program-Library-HOWTO/shared-libraries.html)
162+
163+
164+
5165

0 commit comments

Comments
 (0)