Storage Virtualization

Imagine you're a system administrator managing a database server that's rapidly running out of disk space. The database partition is full, but you have unused space on another partition—or perhaps you've just installed an additional physical disk. In the traditional partitioning model, your options are limited and disruptive: you might need to back up data, repartition the disk, restore data, or even reinstall the operating system. This scenario has plagued system administrators since the dawn of computing.

The Logical Volume Manager (LVM) emerges as an elegant solution to this fundamental problem. LVM introduces a virtualization layer between physical storage devices and the file systems that reside upon them, enabling unprecedented flexibility in storage management. With LVM, enlarging that database partition becomes a simple, online operation—no downtime, no data migration, no system reinstallation.

This page provides an exhaustive exploration of LVM architecture, its design philosophy, the problems it solves, and the fundamental concepts that enable its remarkable flexibility.

To truly appreciate LVM's elegant design, we must first understand the limitations of traditional disk partitioning that motivated its creation. Traditional partitioning, exemplified by MS-DOS MBR and GPT partition tables, directly connects file systems to contiguous regions on physical disks.

The Fixed-Size Dilemma:

When you create a partition using traditional methods, you allocate a fixed, contiguous range of disk sectors to that partition. Once created, the partition's boundaries are essentially immutable. Consider the implications:

• Partition size is determined at creation time, requiring you to predict future storage needs—a notoriously difficult task
• Partitions cannot easily be resized without destructive operations or third-party tools
• Partitions cannot span multiple physical disks, limiting capacity to the size of a single device
• Unused space in one partition cannot be reclaimed by another without repartitioning
• Moving partitions requires copying gigabytes or terabytes of data

The Contiguity Constraint:

Traditional partitions require contiguous disk sectors. This constraint creates fragmentation at the partition level—not within the file system, but in the allocation of partitions themselves. If you have three 100GB partitions and delete the middle one, you have 100GB of free space, but you cannot create a 150GB partition spanning that gap and additional space at the end.

The Single-Disk Limitation:

Perhaps most critically, traditional partitions are confined to a single physical disk. No matter how many disks you add to a system, a single partition cannot leverage that combined capacity. This limitation fundamentally constrains system design and forces administrators into complex workarounds involving symbolic links, mount points, and manual data distribution.

LVM implements a three-layer abstraction that decouples file systems from physical storage devices. This layered architecture is the key to LVM's flexibility and power. Understanding each layer and the relationships between them is essential for mastering storage virtualization.

The Three Fundamental Layers:

1. Physical Volumes (PVs): The lowest layer, representing actual storage devices or partitions
2. Volume Groups (VGs): The middle layer, pooling one or more physical volumes into a unified storage pool
3. Logical Volumes (LVs): The topmost layer, providing block devices that file systems actually use

This architecture follows the classic computer science principle of adding a layer of indirection to solve complexity. By inserting volume groups between physical devices and logical volumes, LVM gains the freedom to map logical storage to physical storage in flexible, non-contiguous ways.

The Abstraction Benefit:

Consider what happens when a file system writes data to a logical volume. The application sees only a standard block device—it has no knowledge that LVM exists. The file system issues I/O requests to the logical volume, which LVM translates into I/O requests to the appropriate physical volume regions. This translation is transparent, automatic, and efficient.

The power of this abstraction becomes apparent when you need to:

• Expand a logical volume: Simply allocate more space from the volume group
• Add more storage: Add a new physical volume to the volume group
• Migrate data: Move physical extents between physical volumes without interrupting access
• Create snapshots: Capture a point-in-time copy without stopping the application

Physical Volumes (PVs) represent the interface between LVM and actual storage hardware. A PV can be created on any block device: a complete disk, a disk partition, a RAID array, or even a loopback device for testing. When you initialize a block device as a PV, LVM writes a special label and metadata area to establish LVM's ownership of that device.

Physical Volume Structure:

A physical volume contains several critical components:

1. LVM Label: A 512-byte structure at the beginning of the device (within the first 4 sectors) containing a UUID, size information, and pointer to metadata areas
2. Metadata Area(s): One or more regions storing the volume group configuration, including all PVs, LVs, and their mappings
3. Physical Extents (PEs): The actual data storage area, divided into fixed-size blocks called extents

Physical Extents (PEs):

Physical extents are the fundamental units of allocation in LVM. When you create a volume group, you specify an extent size (default is 4MB in LVM2). Every physical volume is divided into extents of this size, and every allocation operation works in terms of whole extents.

The extent size represents a crucial trade-off:

• Smaller extents (e.g., 1MB) provide finer-grained allocation but increase metadata overhead and reduce maximum volume size
• Larger extents (e.g., 64MB) reduce metadata overhead but may waste space when logical volumes don't align to extent boundaries

Why 4MB Default?

The 4MB default balances several concerns:

• It's large enough to minimize metadata overhead (one extent entry per 4MB of storage)
• It's small enough to limit wasted space (at most 4MB - 1 byte per logical volume)
• It aligns with common RAID stripe sizes and SSD erase block sizes
• With 4MB extents, a volume group can manage up to 256TB per logical volume (64K extents × 4MB)

LVM2, the current implementation of LVM on Linux, relies on the device-mapper (dm) kernel subsystem to perform the actual I/O remapping. Understanding device-mapper is essential because it's the kernel-level mechanism that makes LVM possible—and device-mapper's capabilities extend far beyond LVM.

What is Device-Mapper?

Device-mapper is a generic framework in the Linux kernel for mapping one block device onto another. It works by creating virtual block devices (dm devices) and intercepting I/O requests to these devices, translating them according to a configurable mapping table, and forwarding them to underlying devices.

Device-mapper provides the foundation for:

• LVM: Logical volume mapping
• dm-crypt: Whole-disk encryption (used by LUKS)
• dm-raid: Software RAID implementation
• dm-cache: SSD caching for HDDs
• dm-thin: Thin provisioning
• dm-snapshot: Copy-on-write snapshots

The Mapping Table:

The heart of device-mapper is the mapping table. Each entry in the table describes how a range of sectors in the virtual device maps to sectors in underlying devices. For a simple linear mapping:

start_sector  num_sectors  target_type  target_args
0             204800       linear       /dev/sda1 0
204800        102400       linear       /dev/sdb1 0

This table creates a virtual device where:

• Sectors 0-204799 map to sectors 0-204799 of /dev/sda1
• Sectors 204800-307199 map to sectors 0-102399 of /dev/sdb1

Target Types:

Device-mapper supports multiple target types:

The core of LVM operation is the mapping between Logical Extents (LEs) in logical volumes and Physical Extents (PEs) in physical volumes. Understanding this mapping mechanism illuminates how LVM achieves its flexibility.

Allocation Policies:

When you create or extend a logical volume, LVM must decide which physical extents to allocate. LVM supports several allocation policies:

1. Contiguous: All LEs must map to physically contiguous PEs on a single PV
2. Cling: New extents should be on the same PV as existing ones if possible
3. Normal (default): Allocate from any available PE, preferring fewer PVs
4. Anywhere: Use any available PE, even from different PVs (enables striping)

These policies allow administrators to balance between:

• Contiguity for performance: Sequential I/O benefits from physically contiguous allocation
• Flexibility for growth: Non-contiguous allocation enables maximum space utilization
• Isolation for reliability: Distributing across PVs can improve fault tolerance

Segment-Based Mapping:

LVM doesn't store a separate mapping entry for every single extent. Instead, it uses segments—contiguous ranges of logical extents that map to contiguous ranges of physical extents on a single PV. This compression dramatically reduces metadata size.

For example, if LEs 0-999 all map to PEs 0-999 on /dev/sda1, LVM stores this as a single segment:

Segment 0: LE 0-999 → /dev/sda1 PE 0-999 (linear, 1000 extents)

Rather than 1000 individual mappings. When the logical volume is extended non-contiguously, a new segment is created:

Segment 0: LE 0-999    → /dev/sda1 PE 0-999  (linear, 1000 extents)
Segment 1: LE 1000-1499 → /dev/sdb1 PE 0-499 (linear, 500 extents)

LVM provides a comprehensive suite of commands for managing physical volumes, volume groups, and logical volumes. Understanding these commands and their underlying operations is essential for effective storage management.

Command Naming Convention:

LVM commands follow a consistent naming pattern:

• pv* commands: Physical volume operations (pvcreate, pvdisplay, pvremove, pvmove)
• vg* commands: Volume group operations (vgcreate, vgdisplay, vgextend, vgreduce)
• lv* commands: Logical volume operations (lvcreate, lvdisplay, lvextend, lvremove)

LVM provides substantial benefits for storage management, but it's not without costs. A thorough understanding of both sides enables informed architectural decisions.

Key Benefits:

This page has provided a comprehensive exploration of the Logical Volume Manager—its architecture, mechanisms, and capabilities. Let's consolidate the essential concepts:

What's Next:

Now that we understand LVM's foundational architecture, the next page dives deep into Volume Groups—the aggregation layer that pools physical storage and enables LVM's flexible capacity management. We'll explore VG creation strategies, extent size selection, metadata handling, and best practices for production deployments.