Description
1 Introduction
In this assignment, you are expected to design and implement a simple file system (SFS) that can be mounted by the user under a directory in the users machine. This assignment along with the supporting files we will provide you will only be functional on Linux. Make sure you don’t waste time coding it on a macOS and trying to run it on Linux later.
2 Some specifications to simplify your life
The SFS introduces many limitations such as restricted filename lengths, no user concept, no protection among files, no support for concurrent access, etc. You could introduce additional restrictions in your design. However, such restrictions should be reasonable to not oversimplify the implementation and should be documented in your submission. Even with the said restrictions, the file system you are implementing is highly useable for embedded applications. The following are the restrictions you will put on your code (this will also help you in coding and simplify your implmentation) : • Max file name of 16 characters. • Max extension name of 3 characters. So in total your file name can have 20 characters (16 characters for name, 1 character for the ”.” period, and finally 3 characters for the extension such as ”txt”) • Single level directory, so there will not be subdirectories (only a single root directory) • You will implement an inode system • You will implement bit map to keep track of free data blocks and find a free data block before increasing the size of a file • The size of one data block is 1024 bytes • The maximum number of data blocks on the disk is 1024
3 About the supporting files
We will provide you with supporting files that include the code structure of solving the assignment. You will edit the C files we give you. This is because will also provide you with test files to test your code. The test code will call functions inside your C file, so please don’t change the main function names. You are free to implement as many helper functions and other C files as you wish (just make sure you edit your Makefile to account for your C files). In fact, through the test files, you will be able to get a sense of what your final grade for this assignment will be!
1
3.1 Emulated disk
Your file system is implemented over an emulated disk system, which is provided to you. The following is a schematic that illustrates the overall concept of the mountable simple file system.
The gray colored modules in the above schematic are provided by the Linux OS. The blue colored modules will be given to you. You are expected to implement the yellow colored module.
The disk emulator given to you provides a constant-cost disk (CCdisk). This CCdisk is a ”virtual” disk space that is separate from your computer disk space. It can be considered as an array of sectors (blocks of fixed size). You can randomly access any given sector for reading or writing. The CCdisk is implemented as a file on the actual file system. Therefore, the data you store in the CCdisk is persistent across program invocations.
To mimic the real disk, the CCdisk is divided into sectors of fixed size. For example, we can split the space into 1024 byte sectors. The number of sectors times the size of a sector gives the total size of the disk. In addition to holding the actual file and directory data, we need to store auxiliary data (meta data) that describes the files and directories in the disk (refer to File attributes section in the lecture notes). The structure and number of bytes spent on meta data storage depends on the file system design, which is the concern in this assignment.
3.2 API for the assignment
As mentioned earlier, the test files we will give you will be calling your C file functions. This means your C file will be an API for the test files. We will give you the API function names and you will have to fill them. The following are the API functions you will implement:
0 void mksfs(int fresh); // creates the file system 1 int sfs_getnextfilename(char *fname); // get the name of the next file in directory 2 int sfs_getfilesize(const char* path); // get the size of the given file 3 int sfs_fopen(char *name); // opens the given file 4 void sfs_fclose(int fileID); // closes the given file 5 void sfs_fwrite(int fileID, char *buf, int length); // write buf characters into disk 6 void sfs_fread(int fileID, char *buf, int length); // read characters from disk into buffer 7 void sfs_fseek(int fileID, int loc); // seek to the location from beginning 8 int sfs_remove(char *file);// removes a file from the filesystem Listing 1: Function names for your C code(you will implement this)
2
The following table descibes how these functions are supposed to behave
Function Explanation mksfs(int fresh) Formats the virtual disk implemented by the disk emulator and creates an instance of the simple file system on top of it. The mksfs() has a fresh flag to signal that the file system should be created from scratch. If flag is false, the file system is opened from the disk (i.e., we assume that a valid file system is already there in the file system). The support for persistence is important so you can reuse existing data or create a new file system. sfs getnextfilename(char *fname) Copies the name of the next file in the directory into fname and returns non zero if there is a new file. Once all the files have been returned, this function returns 0. So, you should be able to use this function to loop through the directory. In implementing this function, you need to ensure that the function remembers the current position in the directory at each call. Remember we have a single-level directory. sfs getfilesize(char *path) Returns the size of a given file. sfs fopen(char *name) Opens a file and returns the index that corresponds to the newly opened file in the file descriptor table. If the file does not exist, it creates a new file and sets its size to 0. If the file does exists, the file is opened in append mode (i.e., set the file pointer to the end of the file). sfs fclose(int fileID) Closes a file. This means you will remove the entry from the open file descriptor table. sfs fwrite(int fileID, char *buf, int length) Writes the given number of bytes of buffered data in buf into the open file, starting from the current file pointer. This in effect could increase the size of the file by the given number of bytes (it may not increase the file size by the number of bytes written if the write pointer is located at a location other than the end of the file). sfs fread(int fileID, char *buf, int length) Reads bytes of data into buf starting from the current file pointer. sfs fseek(int fileID, int loc) Moves the read/write pointer (a single pointer in SFS) to the given location. sfs remove(char *file) Removes the file from the directory entry, releases the file allocation table entries and releases the data blocks used by the file, so that they can be used by new files in the future.
A file system is somewhat different from other OS components because it maintains data structures in memory as well as disk. The disk data structures are important to manage the space in disk and allocate and de-allocate the disk space. Also, the disk data structures indicate where a file is allocated. This information is necessary to access the file. The following is what each function will be doing
3
4 About inode
On-disk data structures of the file system include a super block, the root directory, free block list, and inode table. The figure below shows a schematic of the on-disk organization of SFS.
Figure 1: superblock
The super block defines the file system geometry. It is also the first block in SFS. So the super block needs to have some form of identification to inform the program what type of file system format is followed for storing the data. The figure below shows the proposed structure for the super block. We expect your file system to implement these features, but some modifications are acceptable provided they are well documented. Each field in the figure is 4 bytes long. For instance, the magic number field is 4 bytes long. With a 1024-byte long block , we can see that there will plenty of unused space in the super block.
Figure 2: Super block structure
A file or directory in SFS is defined by an inode. Remember we simplified the SFS by just having a single root directory (no subdirectories). This root directory is pointed to by an inode, which is pointed to by the super block. The inode structure we use here is slightly simplified too. It does not have the double and triple indirect pointers. It has direct and single indirect pointers. With the inode all the meta information (size, mode, ownership) can be associated with the inode. So, the directory entry can be pretty simple. The figure below shows the simplified inode structure.
Figure 3: inode structure
We are suggesting the inode structure shown above to maintain a similarity to the UNIX file system. However, the simplification made to the SFS inode makes it impossible to read or write the SFS using UNIX software or vice-versa.
4
The directory is a mapping table to convert the file name to the inode. Remember a file name can have an extension too. You limit the extension to 3 characters max. A directory entry is a structure that contains two fields (at least): inode and file name. You could add other fields (if you find necessary). Remember the inode also has some attributes such as mode, etc. Depending on the number of entries you have in the directory, the directory could be spanning across multiple blocks in the disk. The inode pointing to the root directory is stored in the super block so we know how to access the root directory. We assume that the SFS root directory would not grow larger than the max file size we could accommodate in SFS.
In addition to the on-disk data structures, we need a set of in-memory data structures to implement the file system. The in-memory data structures improve the performance of the file system by caching the on disk information in memory. Two data structures should be used in this assignment: directory table and inode cache. The directory table keeps a copy of the directory block in memory. Dont make the simplification of limiting the root directory to a single block (this would severely restrict the size of the disk by limiting the number of files in disk). Instead, you could either read the whole directory into the memory or have a cache for the currently used directory block. The later one could be hard to get right.
When you want to create, delete, read, or write a file, first operation is to find the appropriate directory entry. Therefore, directory table is a highly accessed data structure and is a good candidate to keep in memory. Another data structure to cache in the memory is the free block list. In your assignment you will use bit map.
The figure below shows the in-memory data structure and how it connects to the other components. We need at least a table that combines the open file descriptor tables (the per-process one and system-wide one) in a UNIX-like operating system. We simplify the situation because we assume that only one process is accessing a file at any given time.
Figure 4: In-memory data structure
5
When a file is opened we create an entry in this table. The index of the newly created entry is the file descriptor that is returned by the file opening activity. That is the return value of the sfs fopen() is precisely this index.
The entry created in the file descriptor table has at least two pieces of important information: inode number and a read/write pointer. The inode number is the one that corresponds to the file. Remember just like there is an inode for the root directory, there is one inode associated with each file. When a file is opened, that inode number is recorded in this table entry. The read/write pointer is also set according to the file system operating rule. For instance, in this assignment (SFS), you are going to set the read/write pointer to the end of the file at open so that data written into the file will be appended to the file.
In SFS, sfs fseek() is a direct way of setting the read/write pointer value. The interesting problem you could be faced with is what to do when you perform a read or write after setting the read/write pointer. Specifically, if we have a single pointer then sfs fread() would also advance the write pointer, and similarly, the sfs fwrite() would advance the read pointer as well. We can simplify the complexity and let it be that way. You could opt to implement the SFS with two independent read and write pointers as well. In that case, the sfs fseek() needs to have a parameter to specify whether the read, write, or both pointers should be moved by the seek operation.
As shown in the figure above, we have in-memory data structures and on-disk data structures in a file system. The in-memory data structures are activated as soon as the file system is up and running and they are updated every time a file system operation is carried out. While designing and implementing a given file system operation you need to think of the actions that should be carried out on the in-memory and ondisk data structures. In addition to the Open File Descriptor Table, we have variety of different caches for inodes, disk blocks and the root directory. Your design could implement all of them or some of them. File system performance is not a concern for this assignment correct operation is what we need.
5 Suggested steps for your code
5.1 Creating a file • Allocate and initialize an inode. You need to somehow remember the state of the inode table to know which and inode could be allocated for the newly created file. Simply remembering the last inode used is not correct because as you delete files, some inodes in the middle of the table will become unused and available for reuse. • Write the mapping between the inode and file name in the root directory. You could simply update the memory and disk copies. • When no disk data block allocated set the file size to 0. This can also open the file for transactions (read and write). Note that the SFS API does not have a separate create() call. So you can do this activity as part of the open() call.
6
5.2 Writing to a file • Allocate disk blocks (mark them as allocated in your free block list). • Modify the file’s inode to point to these blocks. • Write the data the user gives to these blocks. • Flush all modifications to disk. • Note that all writes to disk are at block sizes. If you are writing few bytes into a file, this might actually end up writing a block to next. So if you are writing to an existing file, it is important you read the last block and set the write pointer to the end of file. The bytes you want to write goes to the end of the previous bytes that are already part of the file. After you have written the bytes, you flush the block to the disk.
5.3 Seek on a file • Modify the read and write pointers in memory. There is nothing to be done on disk.
6 What to submit & test files
We have given you a Makefile, disk emulator (C and Header), SFS test files, and FUSE wrappers. The Makefile shown below has three configurations. The first two tests use hand coded test files to test your implementation. The last test (FUSE wrappers) is a full test that uses kernel functions to manipulate your file system.
0 CFLAGS = -c -g -Wall -std=gnu99 ‘pkg-config fuse –cflags –libs‘ 1 LDFLAGS = ‘pkg-config fuse –cflags –libs‘ 2 3 # Uncomment one of the following lines to run the corresponding scenario 4 #SOURCES= disk_emu.c sfs_api.c sfs_test.c sfs_api.h 5 #SOURCES= disk_emu.c sfs_api.c sfs_test2.c sfs_api.h 6 #SOURCES= disk_emu.c sfs_api.c fuse_wrappers.c sfs_api.h 7 8 OBJECTS=$(SOURCES:.c=.o) EXECUTABLE=First_Lastname_sfs 9 10 all: $(SOURCES) $(HEADERS) $(EXECUTABLE) 11 12 \$(EXECUTABLE): $(OBJECTS) 13 gcc $(OBJECTS) $(LDFLAGS) -o $@ 14 15 .c.o: 16 gcc $(CFLAGS) $< -o $@ 17 clean: 18 rm -rf *.o *~ $(EXECUTABLE)
You will be editing sfs api.c and sfs api.h. So you will be submitting all the files we give. You are free to change the Makefile however you like. This means if you want to structure your code into more C files, that is fine, just remember to edit the Makefile to work approprialty with your code if it needs to.
7
7 Grading
Functionality Grade Illustrated design (pseudo code, UMLs, your favorite strategy). You don’t need to do this if your code is working. If your code does not pass any test, this is a must. 20% File creation, read/write working, able to retrieve previously written data, but does not pass any test. 40% Random reads and writes working with seeking, deletes working and file persistence working 50% Passes test 1 60% Passes test 2 75% Passes Fuse test 90%
Memory leaks handled 95%
Excellent code structure and comments (this will show us your understanding of what functions do, and how your functions work together to achieve the required functionality)
