CDN Caching: Global Content Distribution at Scale

When you stream a video from Netflix, load images on Instagram, or download a software update from Microsoft, you're experiencing the magic of Content Delivery Networks (CDNs). These invisible infrastructure giants ensure that content reaches billions of users worldwide with sub-second latency, regardless of whether the user is in Tokyo, São Paulo, or Cape Town.

Consider the physics of the problem: light travels at approximately 300,000 kilometers per second. A request from Sydney to a server in New York must traverse roughly 16,000 kilometers of fiber optic cable—introducing at least 53 milliseconds of latency each way, 106 milliseconds round-trip, just from the speed of light limitation. In reality, network routing, packet processing, and protocol overhead multiply this to 200-400 milliseconds. For a webpage loading 100 resources, this becomes 20-40 seconds of latency—an eternity in user experience terms.

CDNs solve this fundamental physics problem by eliminating distance.

A Content Delivery Network (CDN) is a geographically distributed network of proxy servers and data centers designed to provide high availability and performance by distributing content closer to end users. Rather than serving all content from a single origin server, CDNs cache and deliver content from edge locations strategically positioned around the world.

The Core Principle: Move Content to Users, Not Users to Content

The fundamental insight behind CDNs is deceptively simple: instead of making every user request travel to a centralized data center, replicate content across geographically distributed locations and serve users from the nearest one. This transforms content delivery from a single-point model to a distributed mesh model.

The Evolution of CDNs

CDNs emerged in the late 1990s as internet traffic began overwhelming origin servers. Akamai, founded in 1998, pioneered the commercial CDN industry with algorithms developed at MIT to solve the "flash crowd" problem—sudden traffic spikes that would crash popular websites. Today, CDNs have evolved from simple static content caches into sophisticated platforms that handle:

• Static content delivery: Images, videos, stylesheets, JavaScript files
• Dynamic content acceleration: API responses, personalized content
• Live and on-demand video streaming: Adaptive bitrate streaming at scale
• Security services: DDoS protection, Web Application Firewalls, bot mitigation
• Edge computing: Running code at edge locations (serverless functions)

A production CDN architecture comprises several interconnected components, each serving a specific function in the content delivery pipeline. Understanding these components is essential for designing effective caching strategies and troubleshooting delivery issues.

The Complete CDN Stack:

Understanding the Request Flow:

1. DNS Resolution: User requests cdn.example.com. The CDN's DNS infrastructure determines the optimal PoP based on geography, network conditions, PoP health, and capacity.

2. Edge Connection: The user connects to the assigned edge PoP. Modern CDNs use persistent connections (HTTP/2, HTTP/3/QUIC) to eliminate connection setup overhead for subsequent requests.

3. Cache Lookup: The edge server checks its local cache for the requested content. This lookup is typically sub-millisecond using in-memory hash tables or SSD-backed storage.

4. Cache Hit (Happy Path): If content is cached and valid (not expired), the edge server responds immediately. Latency is typically 10-50ms depending on user proximity to the PoP.

5. Cache Miss (Origin Fetch): If content is missing or stale, the edge server fetches from the origin shield (if configured) or directly from the origin. The response is cached for subsequent requests.

CDN providers employ different network topologies to balance cost, performance, and operational complexity. Understanding these patterns helps architects make informed decisions about CDN selection and configuration.

Three Primary Topology Patterns:

Mesh Topology (Peer-to-Peer Edge Caching)

Advanced CDNs implement mesh topologies where edge PoPs can fetch content from each other rather than always going to the origin or origin shield. This peer-to-peer approach offers several advantages:

• Reduced Origin Dependency: If PoP-A has content cached, nearby PoP-B can fetch from PoP-A instead of the distant origin
• Improved Resilience: Content availability doesn't depend on a single origin shield
• Lower Latency for Cache Misses: Inter-PoP latency is often lower than PoP-to-origin latency
• Better Long-Tail Content Distribution: Rarely-accessed content can still be served efficiently

The Trade-off Triangle:

Every CDN topology balances three competing concerns:

How does a CDN ensure that a user in Mumbai connects to a server in Mumbai rather than one in Montreal? This is the request routing problem, and CDNs employ multiple sophisticated techniques to solve it.

Primary Routing Mechanisms:

Hybrid Routing: The Best of Both Worlds

Modern CDNs typically combine DNS-based and Anycast routing:

1. Anycast for initial DNS resolution: The CDN's authoritative DNS servers use Anycast for low-latency, resilient DNS resolution
2. GeoDNS/Latency-based routing for edge assignment: DNS responses direct users to optimal edge PoPs using geographic and performance data
3. Anycast at the edge (optionally): Some CDNs also use Anycast for edge server addressing, providing seamless failover

Client-Based Routing (Advanced)

Some CDNs implement client-side routing where JavaScript running in the browser measures latency to multiple PoPs and directs subsequent requests to the fastest one. This provides true end-to-end latency optimization but adds complexity and initial measurement overhead.

Edge servers are the workhorses of CDN infrastructure. Each edge server handles thousands of concurrent connections, performs cache lookups, negotiates TLS, executes edge logic, and routes cache misses—all while maintaining sub-millisecond response times. Understanding their internal architecture illuminates CDN behavior and performance characteristics.

Edge Server Component Stack:

The Cache Storage Hierarchy:

Modern edge servers implement tiered caching to optimize for both speed and capacity:

Cache admission policies determine which tier receives new objects. Simple policies place all objects in L3 and promote based on access frequency. Advanced policies predict object popularity from request patterns to optimize initial placement.

Connection Handling at Scale:

A single edge server may handle 100,000+ concurrent connections. This requires:

• Event-driven I/O: Non-blocking socket operations using epoll (Linux) or kqueue (BSD)
• Connection coalescing: HTTP/2 multiplexing reduces connection count for same-origin requests
• Kernel bypass: DPDK or F-stack moves network processing to userspace for reduced latency
• TLS session resumption: Session tickets and TLS 1.3 0-RTT reduce handshake overhead

How edge servers decide what to cache, what to evict, and where to store objects significantly impacts cache hit rates and overall CDN performance. These decisions involve complex trade-offs between storage capacity, access latency, and operational cost.

Cache Storage Decisions:

Eviction Policy Deep Dive:

LRU (Least Recently Used)

• Evicts the object that hasn't been accessed for the longest time
• Simple to implement with O(1) operations using hash map + doubly linked list
• Works well when recency predicts future access
• Weakness: One-time bulk scans evict hot content (cache pollution)

LFU (Least Frequently Used)

• Evicts objects with the lowest access count
• Retains persistently popular content even if temporarily idle
• Weakness: Historical frequency may not predict future access; new content struggles to gain cache position

SLRU (Segmented LRU)

• Divides cache into probationary and protected segments
• New objects enter probationary segment; re-access promotes to protected
• Protected objects demote to probationary when touched in probationary
• Balances recency and frequency; reduces cache pollution from one-time access

ARC (Adaptive Replacement Cache)

• Maintains four lists tracking recent and frequent access patterns
• Dynamically adapts between LRU and LFU behavior based on workload
• Self-tuning: no manual parameter adjustment required
• Used by ZFS and some advanced caching systems

For mission-critical applications, relying on a single CDN provider introduces unacceptable risk. Multi-CDN architectures distribute traffic across multiple CDN providers to maximize performance, reduce costs, and ensure resilience against provider-specific outages.

Why Multi-CDN?

Multi-CDN Implementation Approaches:

1. DNS-Based Traffic Splitting

• Traffic director (NS1, Route 53, Constellix) routes users to different CDNs based on performance, cost, or load
• Can implement active-passive (primary CDN + failover) or active-active (weighted distribution)
• Limitation: DNS TTLs slow failover; switching CDN doesn't invalidate user's DNS cache

2. Intelligent Traffic Management Platforms

• Platforms like Cedexis (Citrix), Conviva, or Constellix continuously measure CDN performance from global vantage points
• Real-time routing decisions based on actual latency, availability, and throughput metrics
• Can route different user segments to different CDNs (premium users to faster CDN)

3. CDN Load Balancer Layer

• Deploy a CDN specifically for routing to other CDNs
• Example: Cloudflare in front of origin-shielded Akamai and Fastly
• The front CDN becomes the DNS target; it load balances to backend CDNs

4. Application-Layer Routing

• Application logic determines which CDN to use per request
• Maximum flexibility but also maximum complexity
• Useful for dynamic CDN selection based on content type or user tier

We've completed an exhaustive exploration of CDN architecture—from the fundamental physics problem that CDNs solve to the sophisticated components and patterns that enable global content delivery at scale.

Next Steps:

With a solid understanding of CDN architecture, we'll next explore Edge Locations—diving deeper into how CDN providers strategically position their Points of Presence, the infrastructure within each PoP, and how to evaluate CDN coverage for your geographic requirements.