Loadable Kernel Modules

In the earliest days of operating systems, extending kernel functionality meant one thing: recompiling the entire kernel. This was not merely inconvenient—it was operationally catastrophic. Adding support for a new network card driver required shutting down production systems, rebuilding the kernel, and rebooting. In enterprise environments, this meant scheduled downtime, service disruptions, and the ever-present risk that the new kernel wouldn't boot correctly.

Dynamic loading fundamentally transformed this paradigm. It introduced the revolutionary concept that executable code could be inserted into a running kernel—safely, efficiently, and without requiring a reboot. This capability seems almost magical when you first encounter it: the kernel is running, executing critical system code, and yet we can add new code to it while it continues to operate.

Understanding dynamic loading is essential for any serious operating systems engineer. It reveals the sophisticated mechanisms that enable modern kernels to be both stable (by keeping the core minimal and well-tested) and extensible (by supporting thousands of hardware devices and features through loadable modules).

To understand why dynamic loading matters, we must first understand what it replaced: static linking. In a statically linked kernel, all code that will ever execute in kernel space must be determined at compile time and linked into a single, monolithic executable.

The compile-time commitment:

Static linking requires the kernel developer to anticipate every piece of hardware and every feature that any system running this kernel might need. The linker resolves all symbol references—every function call, every global variable access—at compile time, producing a single executable with fixed addresses.

This approach has one significant advantage: performance. Every function call resolves directly to a fixed address. There's no indirection, no symbol lookup at runtime, no relocation overhead. The resulting code is as fast as possible.

But the disadvantages are severe:

The Windows 3.1 case study:

Early versions of Microsoft Windows exemplified the static linking problem. Hardware vendors had to work with Microsoft to include their drivers in the Windows distribution. Installing a new piece of hardware often meant obtaining floppy disks with driver files that the operating system would copy and recognize only after a reboot—and sometimes only after reinstallation.

The UNIX evolution:

Traditional UNIX systems took a different approach. The kernel source was available, and system administrators could rebuild a custom kernel including only the drivers needed for their specific hardware. While this was more efficient than Windows's approach, it required expertise that most users lacked and still demanded downtime for every kernel update.

Dynamic loading is the ability to load executable code into a running program's address space after that program has started execution. When applied to operating system kernels, it enables inserting new kernel code—device drivers, filesystems, network protocols, and more—into the running kernel without rebooting.

This capability rests on three fundamental mechanisms:

The loading workflow:

When the operating system dynamically loads a kernel module, a precise sequence of operations occurs:

1. Read the module file from disk into memory
2. Parse the object file headers to understand the module's structure
3. Allocate kernel memory in a region designated for module code and data
4. Copy code and data sections into the allocated memory
5. Resolve undefined symbols by looking up kernel symbol table entries
6. Apply relocations to adjust addresses in the loaded code
7. Execute initialization code to register the module with kernel subsystems
8. Update kernel data structures to track the loaded module

Memory allocation for modules:

Kernel modules require special memory allocation. Unlike user-space programs that receive virtual address space from the kernel, modules need memory in the kernel's own address space. This memory must be:

• Permanently resident — Module code cannot be swapped to disk, as it may be needed for interrupt handling or when the swap device itself is unavailable
• Executable — The memory region must be marked as containing executable code
• Properly aligned — Many architectures require code to be aligned on specific boundaries
• Contiguous (usually) — While some modern systems support scatter-gather loading, simple loaders expect contiguous allocation

Symbol resolution is the heart of dynamic loading. A kernel module is not a standalone program—it's a fragment of code designed to integrate with the kernel. It calls kernel functions, accesses kernel data structures, and registers itself with kernel subsystems. All these interactions are mediated through symbols.

What is a symbol?

A symbol is a named entity in compiled code—a function, a global variable, or a code label. The compiler generates symbol table entries for each symbol, recording:

• Name — The textual identifier (e.g., printk, kmalloc, current)
• Value — The address or offset where the symbol is located
• Type — Whether it's a function, data object, or other entity
• Binding — Whether it's local (internal) or global (exported)
• Section — Which code or data section contains it

The kernel symbol table:

For modules to reference kernel functions, the kernel must maintain a symbol table of exported symbols. Not all kernel symbols are exported—only those explicitly marked for module use. In Linux, this is done with EXPORT_SYMBOL() and EXPORT_SYMBOL_GPL() macros:

void printk(const char *fmt, ...);
EXPORT_SYMBOL(printk);  // Available to all modules

void internal_kernel_function(void);
// Not exported — modules cannot call this

Symbol lookup during loading:

When a module is loaded, the loader examines its undefined symbols—references to external entities not defined within the module. For each undefined symbol, the loader searches the kernel symbol table:

1. Look up the symbol name in the kernel's exported symbol table
2. If found, record the address for relocation
3. If not found, the load fails with an "unknown symbol" error

Module-to-module dependencies:

Modules can export symbols for use by other modules, creating a dependency graph. For example:

• usbcore exports USB infrastructure symbols
• usb_storage depends on usbcore symbols and exports storage-related symbols
• Specific USB storage device drivers depend on usb_storage symbols

The module loader must process dependencies in the correct order. If module A depends on module B, module B must be loaded first. This is typically resolved through:

• Automatic dependency resolution — The loader reads dependency metadata and loads prerequisites
• Manual specification — The administrator explicitly loads modules in dependency order
• Soft dependencies — Optional dependencies that enhance functionality if present

Relocation is the process of adjusting addresses within code to account for where the code is actually loaded in memory. When a compiler generates an object file, it doesn't know where the code will eventually reside. It uses placeholder addresses or offsets that must be fixed up by the linker (for static linking) or the loader (for dynamic loading).

Why relocation is necessary:

Consider a simple function call:

void my_driver_init(void) {
    printk("Driver initialized
");
}

The compiled code will contain a CALL instruction targeting printk. But at compile time, the address of printk is unknown—it depends on the kernel binary and where the kernel is loaded. The compiler emits a relocation entry recording:

• The location in the code that needs patching
• The symbol being referenced (printk)
• The type of relocation (how to compute the new value)

Where:

• S = Value of the symbol (its address after symbol resolution)
• A = Addend (a constant embedded in the relocation entry)
• P = Place (address of the location being patched)
• L = Address of the PLT entry for the symbol
• G = Offset of the GOT entry for the symbol
• GOT = Address of the Global Offset Table

Relocation processing:

The module loader processes relocations in this sequence:

1. Parse the relocation sections (.rela.text, .rela.data, etc.)
2. For each relocation entry:
   • Resolve the symbol to get its address (S)
   • Calculate the new value using the relocation formula
   • Patch the specified location (P) with the calculated value
3. Repeat until all relocations are processed

Once a module is loaded into memory and all relocations are applied, it's essentially dormant code—present but not active. The initialization entry point brings the module to life, allowing it to register with kernel subsystems, allocate resources, and announce its presence.

The initialization contract:

Kernel modules follow a well-defined contract:

1. Module provides an init function — This function is called exactly once when the module loads
2. Initialization can fail — The function returns 0 on success, or a negative error code on failure
3. On failure, the module is unloaded — Any resources allocated before failure must be cleaned up
4. On success, the module is registered — It becomes part of the running kernel

Finding the entry point:

The loader needs to know which function to call for initialization. This is recorded in the module's metadata:

• ELF note sections — Metadata embedded in the object file
• Special section names — Linux uses .init.text for init code, .exit.text for cleanup
• Symbol names — The loader looks for specifically named symbols (init_module in Linux)

The __init and __exit markers:

Linux uses GCC attributes to optimize module code:

#define __init __attribute__((section(".init.text")))
#define __exit __attribute__((section(".exit.text")))

Code marked __init is placed in a special section that can be discarded after initialization—the function will never be called again, so its memory can be reclaimed. Similarly, __exit code is discarded entirely if the module is compiled into the kernel (since built-in code can never be "unloaded").

The ability to unload a module is as important as loading it. Modules consume kernel memory, and in embedded systems, memory is precious. More critically, unloading enables driver upgrades—replacing an older driver with a newer version without rebooting.

The unloading challenge:

Unloading is far more complex than loading. A loaded module has integrated itself into the kernel:

• It may have registered interrupt handlers
• It may have open file handles
• Other modules may depend on its exported symbols
• Kernel threads may be executing its code
• Hardware may be actively using driver resources

Removing the module while any of these conditions exist would cause system crashes. The unloading process must verify that removal is safe.

The cleanup function:

Every well-designed module provides a cleanup function that reverses everything the initialization function did:

static void __exit my_module_exit(void)
{
    // Unregister from subsystems in reverse order
    unregister_character_device();
    
    // Free allocated resources
    free_device_memory();
    
    // Cancel any pending work
    cancel_delayed_work_sync(&my_work);
    
    printk(KERN_INFO "Module unloaded
");
}

The unloading sequence:

1. Verify the module can be unloaded (reference count, dependencies)
2. Mark the module as "going away" to prevent new references
3. Call the module's cleanup function
4. Wait for any pending operations to complete
5. Remove module symbols from the kernel symbol table
6. Free the memory occupied by module code and data
7. Update module tracking data structures

Kernel modules must coexist with the kernel and other modules in the kernel address space. Understanding how this memory is organized is crucial for both performance and security.

Address space considerations:

In a 64-bit Linux kernel, the virtual address space is typically split:

User space:     0x0000_0000_0000_0000 — 0x0000_7FFF_FFFF_FFFF
                (128 TB of user-accessible memory)

Kernel space:   0xFFFF_8000_0000_0000 — 0xFFFF_FFFF_FFFF_FFFF
                (128 TB of kernel-accessible memory)

Within kernel space, modules are loaded into a specific region. On x86-64 Linux, this is typically around 0xFFFF_FFFF_C000_0000 (the "module area").

Memory protection and W^X:

Modern kernels enforce W^X (Write XOR Execute)—memory that is writable cannot be executable, and vice versa. This is a critical security measure:

• Before relocations: Code sections may be temporarily writable to apply patches
• After relocations: Code sections become read-only + executable
• Data sections: Always writable, never executable

This prevents attackers who corrupt data from injecting executable shellcode.

Module memory allocation:

The kernel provides special allocation functions for module memory:

// Allocate memory for a module
void *module_alloc(unsigned long size);

// Free module memory
void module_memfree(void *ptr);

These differ from regular kmalloc because they allocate from the module region with appropriate permissions, and they may use different page sizes or mapping strategies optimized for code.

Dynamic loading is the architectural foundation that enables modern operating systems to be both lean and extensible. We've explored the mechanisms that make this possible:

What's next:

Now that we understand the foundations of dynamic loading, we'll examine the concrete format used by Linux: the kernel object file format (.ko). We'll see how ELF structures are extended with Linux-specific sections and how the modprobe, insmod, and rmmod tools interact with the kernel loader.