15. Pipes and File Redirection

Creating a Pipe#

#include <unistd.h>

int pipe(int pipefd[2]);

On success:

• pipefd[0] is the read end;
• pipefd[1] is the write end.

]

int pipefd[2];
if (pipe(pipefd) == -1) {
	perror("pipe");
	return 1;
}

The descriptors can be passed to ordinary read() and write(). A pipe contains bytes, not messages. One write of 100 bytes might be returned by several reads, and one read might collect bytes supplied by several writes.

If the bytes represent integers, records or lines, the processes must agree on a protocol: rules for where each piece begins, how its length is known, and what marks the end. A newline-delimited protocol, for example, treats every \n as the end of one record. The pipe transports bytes; your protocol gives those bytes structure.

A Parent Sending Data to a Child#

The usual pattern is to create the pipe before fork(). Both resulting processes inherit both descriptors; each must then close the end it does not use.

#include <stdio.h>
#include <stdlib.h>
#include <sys/wait.h>
#include <unistd.h>

int main(void) {
	int pipefd[2];
	if (pipe(pipefd) == -1) {
		perror("pipe");
		return 1;
	}

	pid_t pid = fork();
	if (pid == -1) {
		perror("fork");
		close(pipefd[0]);
		close(pipefd[1]);
		return 1;
	}

	if (pid == 0) {
		close(pipefd[1]);

		char buffer[128];
		ssize_t n = read(pipefd[0], buffer, sizeof buffer);
		if (n == -1) {
			perror("read");
			_exit(1);
		}
		printf("child received %zd bytes\n", n);
		fflush(stdout);

		close(pipefd[0]);
		_exit(0);
	}

	close(pipefd[0]);
	char message[] = "hello from the parent";
	if (write(pipefd[1], message, sizeof message) == -1) {
		perror("write");
	}
	close(pipefd[1]);

	waitpid(pid, NULL, 0);
	return 0;
}

The shape is:

The complete sequence is:

1. The parent closes its copy of the read end, while the child closes its copy of the write end.
2. The parent writes bytes, then closes the write end when it has no more data.
3. The child reads buffered bytes. Once the buffer is empty and no write descriptor remains open anywhere, its next read() returns 0 for EOF.

This first example performs one small write and one read because it is demonstrating ownership of the two pipe ends. It does not establish a general “one write equals one read” rule. A program which needs the complete message must loop, count the bytes transferred and use its protocol to recognise completion. For example, a fixed 24-byte record is complete only after the reader has accumulated 24 bytes, even if that requires several read() calls.

Why Unused Ends Must Be Closed#

read() on an empty pipe does not immediately mean EOF. It blocks while any descriptor referring to the write end remains open, because another writer could still send bytes.

Suppose the child wants to read until EOF, but accidentally keeps its own inherited write end open. Even after the parent closes its writer, the kernel still sees one writer: the child's unused descriptor. The child waits forever for data which it could theoretically write to itself.

The rule is simple:

• every process closes every pipe end it will not use;
• writers close the write end after their final byte;
• readers treat read() == 0 as end-of-stream.

Closing is part of the communication protocol, not merely cleanup.

Worked EOF Trace#

After fork(), count open write-end descriptors rather than reasoning only from process names:

The child does not receive EOF at the instant the parent calls close(). It first receives any bytes already buffered in the pipe. Only a later read, after the buffer is empty and the writer count is zero, returns 0.

File Descriptor Duplication#

dup() and dup2() make another descriptor refer to the same open file description. This is the descriptor-to-open-file-description distinction introduced in File Systems:

int dup(int oldfd);
int dup2(int oldfd, int newfd);

dup() selects the lowest available descriptor. dup2(oldfd, newfd) uses the requested number, atomically replacing newfd if necessary. If oldfd == newfd, a valid dup2() call changes nothing; it does not close the descriptor and reopen it.

In plain language, dup2(3, 0) means: “make descriptor 0 point wherever descriptor 3 currently points.” It does not copy the file's bytes and it does not move the current position. It changes a table entry. This is exactly why it can turn an ordinary opened file into standard input before execv() starts another program.

The duplicate descriptors share:

• the underlying file or pipe endpoint;
• the current file offset for a regular file;
• file status flags such as append mode.

Read these stages from top to bottom. The descriptor number is the name a process uses; the open-file description is the kernel object which remembers the file and current offset.

1. After open("input.txt", O_RDONLY) — fd 0 still refers to keyboard input. The newly returned fd 3 refers to an input.txt open-file description whose offset begins at zero.
2. After dup2(3, 0) — both fd 0 and fd 3 refer to that same open-file description. Nothing was copied out of the file; descriptor 0 was simply changed to reach the same kernel object as descriptor 3.
3. After close(3) and execv(...) — fd 3 is gone, but fd 0 still refers to input.txt and survives the program replacement. Reads performed by the new program advance the open-file description's offset.

A read through descriptor 0 advances the same shared offset in the underlying open file description. Because the open file description is preserved across execv(), the newly executed program reads seamlessly from input.txt whenever it reads from standard input (STDIN_FILENO).

Practice: Quick Checks#

1. Which element of int pipefd[2] is the read end and which is the write end? Does one write() necessarily correspond to one read()?
2. What does dup2(oldfd, STDOUT_FILENO) do when descriptor 1 is already open?

Redirecting Standard Input and Output#

A program such as tr does not need special "read from a file" logic. It reads descriptor 0. Another process can replace descriptor 0 before executing it:

int fd = open("input.txt", O_RDONLY);
if (fd == -1) {
	perror("input.txt");
	return 1;
}

if (dup2(fd, STDIN_FILENO) == -1) {
	perror("dup2");
	return 1;
}
close(fd);

char *argv[] = {"/usr/bin/tr", "a-z", "A-Z", NULL};
execv(argv[0], argv);
perror("execv");

After dup2, both fd and 0 refer to the same open file description. We close the now-redundant original descriptor, then execv() replaces the program while descriptor 0 survives. The new program reads the file as though it were standard input.

Output redirection follows the same pattern:

int fd = open("message.txt", O_WRONLY | O_CREAT | O_TRUNC, 0666);
if (fd == -1) {
	perror("message.txt");
	return 1;
}

if (dup2(fd, STDOUT_FILENO) == -1) {
	perror("dup2");
	return 1;
}
close(fd);

char *argv[] = {"/bin/echo", "hello", "world", NULL};
execv(argv[0], argv);
perror("execv");

This is equivalent in spirit to echo hello world > message.txt.

Practice: Tracing and Descriptor Lifetime#

3. After fork(), both parent and child hold both pipe ends. The parent closes its write end, but the child keeps its inherited write end while reading until EOF. What happens after the buffered data is consumed?
4. A process opens input.txt as descriptor 3, calls dup2(3, 0), closes 3, then successfully calls execv(). From where does the new program read descriptor 0, and is the underlying offset preserved?

Joining Two Programs with a Pipe#

To implement producer | consumer:

1. create a pipe;
2. create both child processes;
3. in the producer, duplicate the write end onto standard output;
4. in the consumer, duplicate the read end onto standard input;
5. close all original pipe descriptors in every process;
6. execute the respective programs;
7. let the parent wait for both.

The producer and consumer should run concurrently. A pipe's kernel buffer has finite capacity. If the parent waits for the producer before starting the consumer, this sequence can occur:

1. the producer writes until the pipe buffer becomes full;
2. its next write() blocks until some reader makes space;
3. the parent is waiting for that producer to finish;
4. the consumer, which would read and make space, has not been started.

Every participant is waiting for an event which cannot now occur: a deadlock. Start both children first, close the parent's unused pipe ends, and only then wait for them.

Practice: Writing and Debugging#

5. For producer | consumer, list the two dup2() operations and the unused pipe ends each child must close before exec().
6. A parent creates the producer, immediately waits for it, and only afterwards creates the consumer. Explain the finite-buffer failure and state the correct ordering.

Using fdopen#

fdopen() associates a FILE * stream with an existing descriptor, bridging the low-level descriptor interface with the buffered streams from Files and Streams:

FILE *stream = fdopen(pipefd[0], "r");
if (stream == NULL) {
	perror("fdopen");
}

The mode must be compatible with the descriptor's access mode: for example, do not request "w" for a read-only pipe end. You may then use fgets() or fread(). Once the stream owns that descriptor, close it with fclose(stream). Do not separately close the same descriptor and later use the stream.

popen: Convenient but Risky#

FILE *popen(const char *command, const char *mode);
int pclose(FILE *stream);

With mode "r", your process reads the command's standard output. With "w", your process writes to its standard input.

FILE *stream = popen("/bin/date --utc", "r");
if (stream == NULL) {
	perror("popen");
	return 1;
}

char line[256];
if (fgets(line, sizeof line, stream) != NULL) {
	printf("captured: %s", line);
}

int status = pclose(stream);

Close a popen() stream with pclose(), not fclose(), because pclose() also waits for the command. It returns -1 on failure; otherwise its result is an encoded wait status which should be interpreted with macros such as WIFEXITED and WEXITSTATUS, just like the status produced by waitpid().

Like system(), popen() runs a shell command string. It is brittle and vulnerable to command injection if any untrusted text is inserted. Prefer explicit process creation and argument arrays for reliable programs.

Pipes with posix_spawn#

posix_spawn_file_actions_t describes descriptor changes which occur in the child before its new program begins:

#include <errno.h>
#include <spawn.h>

posix_spawn_file_actions_t actions;
int error = posix_spawn_file_actions_init(&actions);
if (error != 0) {
	errno = error;
	perror("posix_spawn_file_actions_init");
	return 1;
}

if ((error = posix_spawn_file_actions_adddup2(
		&actions, pipefd[1], STDOUT_FILENO)) != 0
	|| (error = posix_spawn_file_actions_addclose(
		&actions, pipefd[0])) != 0
	|| (error = posix_spawn_file_actions_addclose(
		&actions, pipefd[1])) != 0) {
	errno = error;
	perror("posix_spawn file action");
	posix_spawn_file_actions_destroy(&actions);
	return 1;
}

error = posix_spawn(&pid, path, &actions, NULL, argv, environ);
int destroy_error = posix_spawn_file_actions_destroy(&actions);

if (error != 0) {
	errno = error;
	perror("posix_spawn");
	return 1;
}
if (destroy_error != 0) {
	errno = destroy_error;
	perror("posix_spawn_file_actions_destroy");
	return 1;
}

The API is verbose, but it avoids constructing a shell command and expresses redirection directly. Always destroy the action object after the spawn attempt and close the parent's copies of descriptors it no longer needs.

Blocking and Broken Pipes#

Pipe behaviour follows the presence of readers and writers:

• reading from an empty pipe blocks while a writer remains;
• reading from an empty pipe returns 0 once all write ends are closed;
• writing may block when the pipe buffer is full;
• writing when no read end remains generates SIGPIPE by default, or fails with EPIPE if the signal is handled or ignored.

This is why descriptor lifetime controls program lifetime. A forgotten writer prevents EOF; a forgotten reader can keep a producer from discovering that its output is unwanted.

Practice: Challenge#

7. What normally happens when a process writes to a pipe with no remaining readers? How does the observable result change if the relevant signal is ignored or handled?
8. Why must a stream returned by popen() be closed with pclose() rather than merely fclose()?