Memory-Mapped Files

Every programmer begins their journey with the traditional I/O model: open a file, read bytes into a buffer, process them, write bytes back, and close. This model is conceptually clean, universally understood, and taught in every introductory programming course. But beneath this simplicity lies a profound inefficiency that becomes painfully apparent at scale.

Consider what happens during a traditional read() call:

1. Your application executes read(fd, buffer, n), triggering a context switch to kernel mode
2. The kernel checks if the requested data exists in its page cache (buffer cache)
3. If not present, the kernel issues a disk I/O operation and blocks your process
4. When data arrives, it's first copied into the kernel's page cache
5. The kernel then copies the data again from page cache to your user-space buffer
6. Control returns to your application with another context switch

This double-copy architecture—from disk to kernel buffer, then from kernel buffer to user buffer—was the accepted paradigm for decades. But what if we could eliminate that second copy entirely? What if your application could access file contents directly through memory operations, as if the file were simply an array in your address space?

The mmap() system call is the POSIX interface for creating memory mappings. Its signature reveals the rich flexibility of the mechanism:

#include <sys/mman.h>

void *mmap(void *addr, size_t length, int prot, int flags, int fd, off_t offset);

At first glance, this six-parameter signature appears daunting. But each parameter addresses a specific aspect of the mapping contract between your application and the operating system. Let's dissect them systematically.

Return Value Semantics:

On success, mmap() returns a pointer to the mapped region—the starting virtual address of the new mapping. On failure, it returns MAP_FAILED (which is (void *) -1) and sets errno to indicate the specific error.

Critical Implementation Detail: Unlike many system calls that return -1 on error, mmap() returns MAP_FAILED. This distinction matters because a NULL return could theoretically be a valid mapping address (though mapping at address 0 is typically prevented by modern kernels for security reasons).

The first parameter, addr, specifies where in your process's virtual address space you want the mapping to appear. This parameter's behavior depends critically on whether you include MAP_FIXED in the flags.

When addr is NULL:

The kernel has complete freedom to choose an appropriate location. This is the most common and recommended usage. The kernel's memory manager will find a suitable hole in your virtual address space, taking into account:

• Existing mappings (avoiding conflicts)
• Architecture-specific constraints (alignment, special regions)
• Address Space Layout Randomization (ASLR) security requirements
• Optimal placement for memory allocator efficiency

When addr is non-NULL without MAP_FIXED:

The address is treated as a hint. The kernel attempts to create the mapping at the specified address, but if that location is unavailable (already mapped, or otherwise unsuitable), it will choose a different address. Your code must always use the returned pointer, never assume the hint was honored.

When addr is non-NULL with MAP_FIXED:

This is a hard requirement. The kernel must place the mapping exactly at the specified address, or fail entirely. If an existing mapping overlaps with the requested region, that existing mapping is silently unmapped—replaced by the new mapping. This behavior is intentional but dangerous.

Modern Alternative: MAP_FIXED_NOREPLACE (Linux 4.17+):

Recognizing the dangers of MAP_FIXED, Linux introduced MAP_FIXED_NOREPLACE. This flag requests exact address placement but fails rather than replacing an existing mapping:

// Safe fixed-address mapping
void *ptr = mmap(
    desired_addr,
    length,
    PROT_READ | PROT_WRITE,
    MAP_PRIVATE | MAP_ANONYMOUS | MAP_FIXED_NOREPLACE,
    -1,
    0
);

if (ptr == MAP_FAILED) {
    if (errno == EEXIST) {
        // Address already mapped - handle gracefully
    }
}

Page Alignment Requirements:

Whether specified as a hint or requirement, the addr parameter (when non-NULL) must be page-aligned. If you specify an address that doesn't fall on a page boundary, the kernel typically rounds down to the nearest page (behavior may vary by implementation). For explicit control, calculate page-aligned addresses:

long page_size = sysconf(_SC_PAGESIZE);  // 4096 on most systems
void *aligned_addr = (void *)((uintptr_t)desired_addr & ~(page_size - 1));

The length parameter specifies the size of the mapping in bytes. The relationship between length and page boundaries is subtle but critical for correct usage.

Automatic Page Rounding:

Internally, the kernel rounds the length up to the nearest page boundary. If you request a 100-byte mapping, you'll actually get at least one full page (typically 4096 bytes). However, accessing bytes beyond your specified length but within the mapped pages is undefined behavior—the kernel may or may not fault on such accesses, leading to non-portable code.

Relationship to File Size:

When mapping files, the interplay between length and file size creates several scenarios:

The SIGBUS Trap:

One of the most confusing aspects of memory-mapped files is the SIGBUS signal. When you access a mapped region that corresponds to bytes beyond the end of the underlying file, the kernel delivers SIGBUS (Bus Error), not SIGSEGV (Segmentation Fault).

Consider this dangerous pattern:

int fd = open("small_file.txt", O_RDONLY);  // File is 100 bytes
void *map = mmap(NULL, 4096, PROT_READ, MAP_PRIVATE, fd, 0);

// This works fine:
char first_byte = ((char *)map)[0];

// This causes SIGBUS, not SIGSEGV:
char byte_1000 = ((char *)map)[1000];  // File only has 100 bytes!

Why SIGBUS Instead of SIGSEGV?

The distinction is architecturally significant:

• SIGSEGV indicates you accessed an address outside any valid mapping—there's no page table entry at all
• SIGBUS indicates the mapping exists (page table entry present), but the underlying physical resource is invalid—you're accessing a page that's mapped but has no backing store

When the accessed offset exceeds the file size, there's no corresponding block on disk. The page is mapped but empty, leading to SIGBUS.

The prot parameter specifies the memory protection for the mapped region. These flags are combined using bitwise OR and determine what operations the CPU will permit on the mapped pages.

Common Combinations:

PROT_READ                    // Read-only mapping (data files)
PROT_READ | PROT_WRITE       // Read-write mapping (modifiable data)
PROT_READ | PROT_EXEC        // Executable code (shared libraries)
PROT_READ | PROT_WRITE | PROT_EXEC  // JIT compilation buffers
PROT_NONE                    // Guard pages, reserved address space

Interaction with File Permissions:

The protection flags must be compatible with how the file was opened:

• PROT_READ requires the file to be opened with O_RDONLY or O_RDWR
• PROT_WRITE with MAP_SHARED requires O_RDWR (writes go to file)
• PROT_WRITE with MAP_PRIVATE only requires O_RDONLY (writes are private, COW)
• PROT_EXEC may have additional restrictions (see below)

If you request protections incompatible with the file descriptor's mode, mmap() fails with EACCES.

PROT_EXEC and W^X (Write XOR Execute) Policy:

Modern security practices enforce W^X: memory should be either writable OR executable, but never both simultaneously. This prevents code injection attacks where an attacker writes malicious code to a buffer and then executes it.

Many operating systems now enforce W^X by default:

• OpenBSD: Strict W^X enforcement, PROT_WRITE | PROT_EXEC fails
• macOS: Requires special entitlements for simultaneous W+X
• Linux: Configurable via SELinux, some distros enforce it
• Windows: DEP (Data Execution Prevention) provides similar protection

For JIT Compilers:

Just-In-Time compilers must work around W^X by:

1. Mapping memory initially as PROT_READ | PROT_WRITE
2. Writing generated code into the pages
3. Using mprotect() to change protection to PROT_READ | PROT_EXEC
4. Then executing the code

// JIT compilation pattern
void *code_buffer = mmap(NULL, size, 
    PROT_READ | PROT_WRITE,
    MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);

generate_machine_code(code_buffer);  // Write code bytes

mprotect(code_buffer, size, PROT_READ | PROT_EXEC);  // Make executable

typedef int (*func_t)(void);
func_t jit_function = (func_t)code_buffer;
int result = jit_function();  // Execute generated code

PROT_NONE: Guard Pages and Address Reservation:

Although PROT_NONE creates pages that cannot be accessed at all, this has important uses:

Guard Pages: Protect against buffer overflows and stack overflow. Place PROT_NONE pages at the boundaries of buffers or around the stack. Any overflow that touches the guard page immediately triggers SIGSEGV, catching the bug early.

// Create a buffer with guard pages
size_t page_size = sysconf(_SC_PAGESIZE);
size_t protected_size = buffer_size + 2 * page_size;

void *region = mmap(NULL, protected_size,
    PROT_NONE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);

// Make middle portion accessible
void *buffer = (char *)region + page_size;
mprotect(buffer, buffer_size, PROT_READ | PROT_WRITE);

// Now 'buffer' is guarded on both ends

Address Reservation: Reserve a large contiguous virtual address region without consuming physical memory. Later, use mprotect() to make portions accessible as needed. This technique is used by custom allocators and garbage collectors to ensure memory growth doesn't require copying data to new locations.

The flags parameter controls the fundamental characteristics of the mapping—whether changes are visible to other processes, whether the mapping backs a file or anonymous memory, and various behavioral modifiers. This is the most complex parameter, with flags organized into categories:

Mandatory Flags (exactly one required):

You must specify exactly one of MAP_SHARED or MAP_PRIVATE. Omitting both, or specifying both, results in undefined behavior (though most implementations treat it as an error).

Optional Modifier Flags:

MAP_ANONYMOUS Deep Dive:

Anonymous mappings don't correspond to any file. Instead, they're backed by the system's swap space (if any) and are initialized to zero. This is actually how most modern memory allocators (malloc implementations) obtain memory from the operating system for large allocations.

// Anonymous private mapping - like malloc() for large blocks
void *heap_block = mmap(NULL, 1024 * 1024,
    PROT_READ | PROT_WRITE,
    MAP_PRIVATE | MAP_ANONYMOUS,
    -1,   // fd must be -1 (or ignored, depending on implementation)
    0     // offset must be 0 (or ignored)
);

Advantages of anonymous mmap over malloc for large allocations:

• Guaranteed page-aligned addresses
• Memory automatically zero-initialized by kernel
• munmap() immediately returns memory to OS (no memory pool fragmentation)
• Explicit control over mapping characteristics

The fd (file descriptor) and offset parameters together specify which file and which portion of that file to map.

The File Descriptor:

For file-backed mappings, fd must be an open file descriptor with appropriate access rights:

int fd = open("data.bin", O_RDWR);  // For MAP_SHARED with PROT_WRITE
int fd = open("data.bin", O_RDONLY); // Sufficient for MAP_PRIVATE even with PROT_WRITE

File Descriptor Lifecycle:

A commonly misunderstood point: the file descriptor can be closed immediately after mmap() returns. The mapping maintains its own reference to the underlying file (technically, to the struct file in kernel space). This means:

int fd = open("file.txt", O_RDONLY);
void *map = mmap(NULL, size, PROT_READ, MAP_PRIVATE, fd, 0);
close(fd);  // Perfectly safe! Mapping remains valid.

// Continue using 'map' for as long as needed
printf("%s", (char *)map);

munmap(map, size);  // Clean up when done

This behavior enables cleaner code—you don't need to track file descriptors for the lifetime of mappings.

For Anonymous Mappings:

When using MAP_ANONYMOUS, the fd parameter should be -1. Some older Unix systems require fd to be -1; others ignore it. For maximum portability, always pass -1.

The offset Parameter:

The offset specifies where within the file the mapping should begin. Two critical requirements:

1. Must be page-aligned: The offset must be a multiple of the system page size. Non-page-aligned offsets cause mmap() to fail with EINVAL.

2. Affects file positioning: The mapping begins at file position offset and extends for length bytes.

Mapping Beyond File End:

If offset + length exceeds the file size at the time of mmap(), the behavior depends on how far beyond:

• Within the last mapped page: The kernel zero-fills bytes between EOF and the page boundary. Accessing these zeros is safe (reads return zero, writes to MAP_PRIVATE work but are discarded on munmap, writes to MAP_SHARED may extend the file).

• Beyond the last mapped page: Accessing these addresses triggers SIGBUS.

Windows Comparison:

On Windows, the equivalent is CreateFileMapping() + MapViewOfFile(). While the concepts are similar, Windows uses a two-step process and has different parameter semantics. Cross-platform code must abstract these differences.

When mmap() fails, it returns MAP_FAILED and sets errno to indicate the specific error. Understanding these errors is essential for robust code:

Understanding what happens inside the kernel during an mmap() call illuminates why certain behaviors exist and helps predict performance characteristics.

The Virtual Memory Area (VMA):

When mmap() succeeds, the kernel creates a Virtual Memory Area (VMA) structure that describes the mapping. In Linux, this is struct vm_area_struct. Each VMA contains:

• Start and end addresses of the mapping
• Protection flags (translated to architecture-specific page table bits)
• Flags (shared/private, etc.)
• Pointer to the mapped file (if file-backed)
• Offset within the file
• Operations table (for handling page faults, etc.)

What mmap() Does NOT Do:

Critically, mmap() typically does not:

1. Allocate physical RAM: No physical pages are allocated for the mapping (unless MAP_POPULATE is specified)
2. Read file contents: No I/O occurs to fetch file data
3. Create page table entries: Pages are not mapped in hardware page tables

Mmap() merely creates the VMA—a promise that "if you access addresses in this range, the kernel knows what to do." The heavy lifting is deferred.

Page Fault Handling:

The deferred nature means that actual I/O happens during page faults:

1. Application accesses address within the mapped region
2. CPU consults page table—no entry exists (or entry is marked not-present)
3. CPU raises a page fault exception
4. Kernel's page fault handler runs:
   • Finds the VMA containing the faulting address
   • Determines this is a valid access (not a bug/attack)
   • For file-backed: checks page cache for this file+offset
   • If not cached: allocates physical page, issues disk I/O
   • Creates page table entry mapping virtual → physical
5. Returns from exception; CPU retries the faulting instruction
6. Access now succeeds

This demand-paging approach means:

• Starting a process is fast (don't need to load entire executable)
• Memory is allocated only when actually used
• Unused portions of mapped files consume zero RAM

Page Cache Sharing:

The page cache is the key to mmap efficiency. When multiple processes map the same file:

• The kernel recognizes they're mapping the same file (via inode)
• All maps share the same physical pages in the page cache
• Only one copy exists in RAM, regardless of how many processes use it

This is why shared libraries (.so/.dll files) are so efficient—the code pages exist once in physical memory, mapped into potentially thousands of processes.

Every mmap() should eventually be matched with a munmap() to release the mapping. The signature is straightforward:

#include <sys/mman.h>

int munmap(void *addr, size_t length);

Parameters:

• addr: Starting address of the mapping (as returned by mmap())
• length: Size of the region to unmap

Return Value:

• Returns 0 on success
• Returns -1 on failure, with errno set

Important Considerations:

We've comprehensively examined the mmap() system call—the fundamental interface for memory-mapped files and anonymous memory allocation. Let's consolidate the key points:

What's Next:

With the mmap() interface mastered, we'll explore how memory-mapped files transform file access patterns. The next page examines the "File as Memory" paradigm—how treating files as byte arrays eliminates the read/write system call overhead and enables elegant solutions to complex problems.