14. File System Metadata and Directories

File System Metadata and Directories#

File Systems explains how a process obtains descriptors and transfers bytes. This note looks one layer outward: how those byte sequences are named, organised and protected.

At a useful first approximation:

• a regular file is a sequence of bytes;
• a directory maps names to file-system objects;
• an inode stores metadata and points towards stored contents;
• a pathname tells the OS which directory entries to traverse.

The filename is therefore not the file itself. It is an entry in a directory which leads us to an inode.

Paths#

An absolute path begins at the root directory /, while a relative path begins at the process's current working directory:

/usr/include/stdio.h       absolute
../../another/program.c    relative
./a.out                    relative
main.c                     relative

If the current working directory is /home/z5555555/lab07 and a program opens main.c, the path resolves to /home/z5555555/lab07/main.c.

Two names are special inside directories:

• . refers to the current directory;
• .. refers to its parent.

Unix filenames may contain almost any non-zero byte except /: zero terminates a C string and / separates path components. A leading . is only a convention interpreted by programs such as ls and the shell.

flowchart TD
    ROOT["/"] --> HOME["home"]
    ROOT --> USR["usr"]
    HOME --> USER["z5555555"]
    USER --> LAB["lab07"]
    LAB --> MAIN["main.c"]
    USR --> INCLUDE["include"]
    INCLUDE --> STDIO["stdio.h"]

The hierarchy looks like a tree until symbolic links are followed. A symlink can point elsewhere in the hierarchy, creating a graph and even a cycle.

Inodes and Directory Entries#

An inode stores metadata for one file-system object, including:

• object type: regular file, directory, symbolic link, device and so on;
• size in bytes;
• owner user ID and group ID;
• permission bits;
• timestamps and link count;
• information used to locate the object's data.

A directory roughly contains pairs of (name, inode number). Multiple directory entries can refer to one inode, which is how hard links work.

flowchart TD
    subgraph D1["Directory: /home/z5555555/docs"]
        E1["report.txt | inode 4812"]
        E2["summary.txt | inode 4812"]
        E3["pic.png | inode 9104"]
    end

    subgraph D2["Directory: /home/z5555555/backup"]
        E4["old_report.txt | inode 4812"]
    end

    subgraph IT["Filesystem Inode Table"]
        I1["Inode 4812<br/>type: regular file<br/>size: 4096 bytes<br/>permissions: 0644<br/>hard links: 3<br/>blocks: [102, 103]"]
        I2["Inode 9104<br/>type: regular file<br/>size: 20480 bytes<br/>permissions: 0600<br/>hard links: 1<br/>blocks: [205, 206, 207]"]
    end

    subgraph DB["Storage Data Blocks"]
        B1["Blocks 102, 103<br/>(Text contents of report)"]
        B2["Blocks 205..207<br/>(PNG image bytes)"]
    end

    E1 --> I1
    E2 --> I1
    E4 --> I1
    E3 --> I2

    I1 --> B1
    I2 --> B2

In the diagram above, report.txt, summary.txt, and old_report.txt are three hard links that share inode 4812. The inode's link count is 3. Removing report.txt simply deletes that directory entry and decrements the link count to 2; the underlying data blocks remain intact until the link count reaches 0 and no open file descriptions reference it.

Permissions#

Unix permissions are divided between user (the owner), group and other. Each has read, write and execute bits:

The same bit has a different practical meaning for a file and a directory:

Without execute permission on a directory, knowing an entry's name does not let a process traverse through that directory to access it. In practice, creating or removing an entry normally requires both write and execute permission on its parent directory. Sticky-bit rules and permissions on surrounding directories can impose further restrictions.

Octal Permissions#

Each group of three permission bits becomes one octal digit. This is the same mask-and-test model developed in Bitwise Operations, applied to metadata rather than an ordinary program value:

Therefore 0755 is rwxr-xr-x, while 0640 is rw-r-----.

chmod 640 report.txt

The leading zero in a C integer such as 0640 means octal. When an object is created, the process's umask may remove some requested permissions.

Inspecting Metadata with stat#

#include <sys/stat.h>

int stat(const char *path, struct stat *result);
int lstat(const char *path, struct stat *result);
int fstat(int fd, struct stat *result);

• stat() follows a symbolic link and reports its target;
• lstat() reports the symbolic link itself;
• fstat() reports the object already referred to by an open descriptor.

struct stat info;
if (lstat(path, &info) == -1) {
	perror(path);
	return 1;
}

printf("inode: %llu\n", (unsigned long long)info.st_ino);
printf("size: %lld bytes\n", (long long)info.st_size);
printf("links: %llu\n", (unsigned long long)info.st_nlink);

st_mode combines type and permission bits. Use the macros rather than raw magic values:

if (S_ISREG(info.st_mode)) {
	puts("regular file");
} else if (S_ISDIR(info.st_mode)) {
	puts("directory");
} else if (S_ISLNK(info.st_mode)) {
	puts("symbolic link");
}

if (info.st_mode & S_IXOTH) {
	puts("others may execute or traverse it");
}

S_ISLNK can only succeed when the metadata belongs to the link itself, normally through lstat().

Hard Links#

A hard link is another directory entry for the same inode:

ln original.txt another-name.txt

The two names have equal status. Changing the bytes through either path changes the same file. Removing one name only removes that directory entry; storage remains while another hard link or open descriptor still refers to the inode.

Hard links normally cannot cross file-system boundaries, because inode numbers belong to one file system. Ordinary users also cannot hard-link directories, avoiding cycles in the directory hierarchy.

Symbolic Links#

A symbolic link, or symlink, is a separate object whose contents name another path:

ln -s original.txt shortcut.txt

A symlink can point to a directory, cross file-system boundaries, become dangling if its target disappears, and participate in cycles.

flowchart LR
    H1["hard-link name A"] --> I["same inode"]
    H2["hard-link name B"] --> I
    S["symlink inode<br/>contains path 'name A'"] -."path lookup".-> H1

A relative target stored in a symlink is interpreted relative to the directory containing the symlink, not the working directory of whichever process follows it.

Creating and Reading Directories#

#include <sys/stat.h>

if (mkdir("new-directory", 0755) == -1) {
	perror("mkdir");
}

mkdir("a/b/c", 0755) does not create missing parents. Both a and a/b must already exist.

Directory traversal uses a directory stream:

#include <errno.h>
#include <dirent.h>
#include <stdio.h>

DIR *directory = opendir(path);
if (directory == NULL) {
	perror(path);
	return 1;
}

for (;;) {
	errno = 0;
	struct dirent *entry = readdir(directory);
	if (entry == NULL) {
		if (errno != 0) {
			perror("readdir");
		}
		break;
	}

	printf("%s\n", entry->d_name);
}

if (closedir(directory) == -1) {
	perror("closedir");
}

readdir() also yields . and ... Its pointer refers to library-managed storage which a later call may overwrite, so copy a name if it must be retained.

Do not rely solely on d_type; some file systems report DT_UNKNOWN. Construct the full path and call lstat() when the type matters:

char full_path[4096];
int length = snprintf(full_path, sizeof full_path, "%s/%s", path, entry->d_name);
if (length < 0 || (size_t)length >= sizeof full_path) {
	fprintf(stderr, "path is too long\n");
	// Do not use the truncated pathname.
}

Other Useful Operations#

int chmod(const char *path, mode_t mode);
int unlink(const char *path);
int rename(const char *old, const char *new);
int chdir(const char *path);
char *getcwd(char *buffer, size_t size);
int link(const char *old, const char *new);
int symlink(const char *target, const char *linkpath);

unlink() is an accurate name: it removes one connection between a directory name and an inode. The storage can be reclaimed after the last hard link is removed and the last open reference is closed.

"Everything is a File"#

Unix extends the descriptor interface beyond stored files. Terminals, devices, pipes and sockets can all be accessed through file-like operations. The slogan does not claim that they are ordinary disk files; it means the same small read/write interface can describe many resources.

This composability is what allows Pipes and File Redirection to work. A program reading descriptor 0 need not know whether its bytes come from a keyboard, a regular file or another process.

Common Mistakes#

• Thinking a filename lives inside the inode. Names live in directories.
• Treating one hard link as the original and another as merely a shortcut.
• Assuming removing a symlink removes its target.
• Using stat() when the symlink itself is the object of interest.
• Treating directory permissions as though they mean the same as regular-file permissions.
• Recursively following symlinks without cycle detection.
• Forgetting to skip . and .. during recursion.

Practice#

1. What is the difference between stat() and lstat()?
2. What happens to a file's data blocks when you call unlink() on a filename that still has two other hard links?
3. Why does directory execute permission (+x) matter even if you have read permission (+r) on that directory?
4. Why can't hard links cross file-system boundaries, whereas symbolic links can?
5. When traversing a directory with readdir(), why must recursive code explicitly check for . and ..?