45:"/_next/static/chunks/33yus346z0l_n.js" 46:"/_next/static/chunks/2j22cxchii_xx.js" 47:"/_next/static/chunks/0iv56yrji_20j.js" 48:"/_next/static/chunks/32izpn90gvm22.js" 49:"/_next/static/chunks/3j8bi-zu9bqs4.js" 4a:"/_next/static/chunks/1z-pox7hqp2x3.js" 4b:"/_next/static/chunks/20w4eaqlekefm.js" 4c:"/_next/static/chunks/1tn-0fivsxsv1.js" 4d:"/_next/static/chunks/09nq57stwzkgf.js" 4e:"/_next/static/chunks/3m1xjj5kw5fie.js" 4f:"/_next/static/chunks/0gafyzg9j9yqj.js" 50:"/_next/static/chunks/3k2gyg36fnag_.js" 51:"/_next/static/chunks/3bgoyiso1rhn7.js" 52:"/_next/static/chunks/3cstd6xzfli_o.js" 53:"/_next/static/chunks/253gnmj0-cj17.js" 54:"/_next/static/chunks/0hlb44ca7-4a2.js" 55:"/_next/static/chunks/20329i6m_04t6.js" 56:"/_next/static/chunks/40mq7ggq-ozo4.js" 57:"/_next/static/chunks/082jo-0ajazgr.js" 58:"/_next/static/chunks/2ozcrjb_hh74t.js" 59:"/_next/static/chunks/01a-3hvnqu8wl.js" 5a:"/_next/static/chunks/05i8en-x14ibi.js" 5b:"/_next/static/chunks/3m0r7x0jt5wm7.js" 5c:"/_next/static/chunks/2k8pepwc4pxns.js" 5d:"/_next/static/chunks/0wa9m1-tphf9u.js" 5e:"/_next/static/chunks/30jdm3dwz834g.js" 5f:"/_next/static/chunks/0k9bl6dipnsvu.js" 60:"/_next/static/chunks/18ggtgz2yf759.js" 61:"/_next/static/chunks/0i3-eynrkvqi7.js" 62:I[593304,["$2a","$2b","$2c","$2d","$2e","$2f","$45","$46","$47","$48","$49","$4a","$4b","$4c","$4d","$4e","$4f","$50","$51","$52","$53","$54","$55","$56","$57","$58","$59","$5a","$5b","$5c","$5d","$5e","$5f","$60","$61"],"OwnedHydration"] 63:T4d6,**Parallelization** is the technique of dividing work into smaller units that can be executed simultaneously across multiple processing resources. Unlike sequential execution where tasks complete one after another, parallel execution exploits the availability of multiple CPUs, cores, threads, or machines to process work concurrently. **The fundamental insight:** Modern computing resources are massively parallel. A single server might have 64+ CPU cores, each capable of independent computation. A distributed system might span thousands of machines. Sequential processing utilizes only a tiny fraction of this capacity. Parallelization unlocks the full potential. **Amdahl's Law—The Theoretical Limit:** Amdahl's Law defines the theoretical speedup achievable through parallelization: $$ \text{Speedup} = \frac{1}{(1 - P) + \frac{P}{N}} $$ Where: - **P** = Proportion of the program that can be parallelized (0 to 1) - **N** = Number of parallel processing units **Key implications:** - If only 90% of work is parallelizable (P=0.9), infinite processors yield at most 10x speedup - Serial portions become the dominant constraint as parallel resources increase - Maximizing P is often more impactful than adding more processors64:T6ea,**Process-Based Parallelism:** Processes provide the strongest isolation—each process has its own memory space, file descriptors, and runtime state. A crash in one process doesn't affect others (hence "crash isolation"). ``` ┌─────────────────┐ │ Parent Process │ │ ┌───────────┐ │ │ │ Memory │ │ │ └───────────┘ │ └────────┬────────┘ │ ┌────────────────┼────────────────┐ │ │ │ ┌──────▼──────┐ ┌──────▼──────┐ ┌──────▼──────┐ │ Worker 1 │ │ Worker 2 │ │ Worker N │ │ ┌─────────┐ │ │ ┌─────────┐ │ │ ┌─────────┐ │ │ │ Memory │ │ │ │ Memory │ │ │ │ Memory │ │ │ └─────────┘ │ │ └─────────┘ │ │ └─────────┘ │ └─────────────┘ └─────────────┘ └─────────────┘ (Isolated) (Isolated) (Isolated) ``` **When to use processes:** - CPU-bound workloads (bypass Python GIL, etc.) - Untrusted or potentially crashing code - Multi-language architectures - Microservices (each service is a process)65:T74b,**Thread-Based Parallelism:** Threads share memory within a process, enabling efficient data sharing but requiring careful synchronization to avoid race conditions. ``` ┌───────────────────────────────────┐ │ Single Process │ │ ┌──────────────────────────────┐ │ │ │ Shared Memory │ │ │ │ (Heap, Global Variables) │ │ │ └──────────────────────────────┘ │ │ │ │ ┌───────┐ ┌───────┐ ┌───────┐ │ │ │Thread1│ │Thread2│ │ThreadN│ │ │ │ Stack │ │ Stack │ │ Stack │ │ │ └───────┘ └───────┘ └───────┘ │ └───────────────────────────────────┘ ``` **Thread safety considerations:** - Mutable shared state requires synchronization (locks, mutexes) - Lock contention can serialize parallel execution - Deadlocks possible with multiple locks - Thread-local storage avoids sharing overhead **Languages and thread models:** - Java: Full threading with synchronized blocks, concurrent collections - Go: Goroutines (lightweight threads) with channels for communication - Python: Limited by GIL for CPU-bound work; effective for I/O-bound - Rust: Ownership model prevents data races at compile time66:T65b,**Coroutines and Async/Await:** Coroutines provide concurrency without parallelism—multiple tasks share a single thread, yielding cooperatively at I/O boundaries. This enables handling thousands of concurrent connections with minimal overhead. ``` ┌────────────────────────────────────────────┐ │ Single Thread │ │ │ │ Request 1: ●→→→→○ (waiting for I/O) │ │ ↓ yield │ │ Request 2: ●→→→○ (waiting) │ │ ↓ yield │ │ Request 3: ●→→→→○ │ │ ↓ yield │ │ Request 1: (I/O complete) →→→→● │ │ │ │ ● = executing ○ = blocked/waiting │ └────────────────────────────────────────────┘ ``` **Key insight:** While coroutines don't provide true CPU parallelism, they dramatically increase throughput for I/O-bound workloads by ensuring the CPU is always doing useful work instead of waiting for I/O. **Example concurrency capacity:** - Traditional threaded server: ~1,000-10,000 concurrent connections - Async/coroutine server: ~100,000-1,000,000 concurrent connections67:T710,**Scatter-Gather Pattern Deep Dive:** This pattern is ubiquitous in microservices architectures where a single request requires data from multiple services. ``` Request │ ▼ ┌─────────────┐ │ Coordinator │ │ (Scatter) │ └──────┬──────┘ │ ┌──────┼──────┬──────┬──────┐ │ │ │ │ │ ▼ ▼ ▼ ▼ ▼ ┌──────┐┌──────┐┌──────┐┌──────┐┌──────┐ │Svc A ││Svc B ││Svc C ││Svc D ││Svc E │ └──┬───┘└──┬───┘└──┬───┘└──┬───┘└──┬───┘ │ │ │ │ │ └───────┴───────┼───────┴───────┘ │ ┌───────▼───────┐ │ Coordinator │ │ (Gather) │ └───────┬───────┘ │ ▼ Response ``` **Critical considerations:** 1. **Timeout strategy:** What happens when one service is slow? - Wait indefinitely → Entire request blocked - Strict timeout → Partial results or error - Best practice: Return partial results with degraded experience 2. **Failure handling:** One service fails → fail entire request? Return partial? 3. **Result ordering:** Results arrive out of order; aggregation must handle this 4. **Connection management:** N parallel calls = N connections; need connection pooling68:T761,// Scatter-Gather pattern with timeout and partial results interface ServiceResult { service: string; success: boolean; data?: T; error?: string; latencyMs: number; } async function scatterGather( services: string[], fetcher: (service: string) => Promise, timeoutMs: number = 200 ): Promise[]> { const fetchWithTimeout = async (service: string): Promise> => { const start = Date.now(); const timeoutPromise = new Promise((_, reject) => setTimeout(() => reject(new Error('Timeout')), timeoutMs) ); try { const data = await Promise.race([ fetcher(service), timeoutPromise ]); return { service, success: true, data, latencyMs: Date.now() - start }; } catch (error) { return { service, success: false, error: error instanceof Error ? error.message : 'Unknown error', latencyMs: Date.now() - start }; } }; // Scatter: Launch all fetches in parallel const promises = services.map(fetchWithTimeout); // Gather: Wait for all to complete (or timeout) const results = await Promise.allSettled(promises); // Aggregate results return results .filter((r): r is PromiseFulfilledResult> => r.status === 'fulfilled' ) .map(r => r.value); } // Usage example: Fetch user dashboard data from multiple microservices const dashboardData = await scatterGather( ['user-service', 'recommendations', 'notifications', 'analytics'], async (service) => { const response = await fetch(`http://${service}/api/data/${userId}`); return response.json(); }, 150 // 150ms timeout ); const successfulResults = dashboardData.filter(r => r.success); console.log(`Got ${successfulResults.length} of 4 services within timeout`);69:T47d,**Database Query Parallelization:** Modern databases parallelize query execution across multiple dimensions: - **Intra-Query Parallelism:** A single query uses multiple threads - Parallel table scans (each thread reads different blocks) - Parallel index lookups - Parallel aggregation (compute partial GROUP BY in parallel, merge at end) - Parallel sorts (parallel quicksort, then merge) - **Inter-Query Parallelism:** Multiple queries execute concurrently - Connection pool serving multiple clients - Read queries can run fully parallel (no locks for snapshot isolation) - Write queries require coordination (transaction isolation) **PostgreSQL Parallel Query Example:** ```sql -- PostgreSQL will use parallel workers for this query SET max_parallel_workers_per_gather = 4; EXPLAIN ANALYZE SELECT category, SUM(amount) FROM transactions WHERE created_at > '2024-01-01' GROUP BY category; -- Output shows: Gather Merge -> Parallel Hash Aggregate -> Parallel Seq Scan ``` PostgreSQL's planner automatically parallelizes when: - Table is large enough to benefit - Query type supports parallelism - Parallel workers are available6a:T69e,**Web Server Parallelization Models:** **1. Process-Per-Request (Apache prefork):** - Each request handled by a separate process - High isolation, high memory cost - Scales to ~thousands of concurrent requests **2. Thread-Per-Request (Apache worker, Tomcat):** - Thread pool serves incoming requests - Shared memory reduces overhead - Scales to ~10,000 concurrent requests **3. Event-Driven (NGINX, Node.js):** - Single or few threads handle many connections - Non-blocking I/O with event notifications - Scales to ~100,000+ concurrent connections **4. Hybrid (Gunicorn + uvloop):** - Multiple worker processes (for multi-core) - Each runs async event loop (for high concurrency) - Scales to ~100,000+ per process × number of processes **NGINX Architecture:** ``` ┌─────────────────────────────────────────┐ │ Master Process │ │ (Configuration, process management) │ └───────────────┬─────────────────────────┘ │ ┌───────────┼───────────┐ │ │ │ ▼ ▼ ▼ ┌────────┐ ┌────────┐ ┌────────┐ │Worker 1│ │Worker 2│ │Worker N│ │ (100K │ │ (100K │ │ (100K │ │ conns) │ │ conns) │ │ conns) │ └────────┘ └────────┘ └────────┘ ``` Each worker handles tens of thousands of connections via epoll/kqueue.6b:T668,**Parallel Data Processing Frameworks:** **Apache Spark:** - Partitions data across cluster nodes - Each partition processed independently - Shuffle operations for joins/aggregations ``` Dataset → Partitions → Tasks → Executors │ │ │ │ │ │ │ ▼ │ │ │ [Core 1][Core 2]... │ │ │ ↓ │ │ └── One task per partition │ └────────────── Data divided by key/range └──────────────────────── Logical data abstraction Throughput = (Partitions × Cores × Processing Speed) / Shuffle Overhead ``` **Kafka Consumer Groups:** - Topic partitions assigned to consumers - Each partition consumed by exactly one consumer - Adding consumers parallelizes consumption (up to partition count) ``` Topic: user-events (12 partitions) │ ┌───────────────┼───────────────┐ │ │ │ ┌───▼───┐ ┌───▼───┐ ┌───▼───┐ │Consumer│ │Consumer│ │Consumer│ │ 1 │ │ 2 │ │ 3 │ │P0,P1, │ │P4,P5, │ │P8,P9, │ │P2,P3 │ │P6,P7 │ │P10,P11 │ └────────┘ └────────┘ └────────┘ 3 consumers = 3x parallel consumption ```43:["$","$L62",null,{"owner":"anonymous","state":{"mutations":[],"queries":[{"dehydratedAt":1792306257326,"state":{"data":{"json":{"id":"seed_3f1125d5754a425e28726c390c3f3c4d","slug":"throughput-optimization","category":"System Design (HLD)","topic":"Throughput Optimization","title":"Throughput Optimization Techniques","description":"Master the art of increasing system throughput through parallelization, batching, connection reuse, queue-based processing, and horizontal scaling. Learn to design systems that handle massive volumes of concurrent requests efficiently.","difficulty":"INTERMEDIATE","estimatedMinutes":120,"pages":[{"id":"seed_2e6d59deded6fb092aacfd4ca99e1860","title":"Parallelization: Maximizing Concurrent Execution","order":0,"content":[{"id":"opening-hook","title":"The Throughput Imperative","content":[{"data":{"content":"When engineers discuss system performance, they often conflate two fundamentally different metrics: **latency** (how fast a single request completes) and **throughput** (how many requests the system can handle per unit of time). While latency optimization focuses on making individual operations faster, **throughput optimization** is about maximizing the total work accomplished—the volume of requests processed, data transformed, or transactions completed.\n\nConsider two database systems:\n- **System A:** Processes each query in 10ms, but can only handle 100 queries at a time = 10,000 queries/second\n- **System B:** Processes each query in 20ms, but can handle 1,000 concurrent queries = 50,000 queries/second\n\nSystem B has worse per-query latency but **5x higher throughput**. For many workloads—batch processing, analytics pipelines, high-traffic APIs—throughput is the primary constraint, not latency."},"type":"text"},{"data":{"icon":"Target","title":"What You Will Learn","content":"This page deep-dives into parallelization—the most fundamental throughput optimization technique. You'll understand different parallelization models, their trade-offs, implementation strategies, and when each approach is appropriate. By the end, you'll be able to design systems that extract maximum throughput from available hardware through intelligent parallel execution.","variant":"info"},"type":"callout"}]},{"id":"understanding-parallelization","title":"Understanding Parallelization","content":[{"data":{"content":"$63"},"type":"text"},{"data":{"rows":[["50%","1.33x","1.78x","1.97x","2.00x"],["75%","1.60x","2.91x","3.77x","4.00x"],["90%","1.82x","4.71x","8.89x","10.00x"],["95%","1.90x","5.93x","14.69x","20.00x"],["99%","1.98x","7.48x","39.26x","100.00x"]],"caption":"Speedup Under Amdahl's Law (Maximum Theoretical)","headers":["Parallelizable (P)","2 Cores","8 Cores","64 Cores","Infinite"]},"type":"table"},{"data":{"icon":"AlertCircle","title":"The Serial Bottleneck Trap","content":"Many teams add more servers expecting linear throughput gains, only to discover serialized components (shared locks, single-threaded coordinators, strict ordering requirements) cap their improvement. Before scaling out, identify and eliminate serial bottlenecks—this often yields greater returns than adding hardware.","variant":"warning"},"type":"callout"}],"titleType":"number","titleNumber":1},{"id":"types-of-parallelism","title":"Types of Parallelism","content":[{"data":{"content":"Parallelism manifests in different forms depending on what is being parallelized and how. Understanding these distinctions is critical for choosing the right approach for your workload."},"type":"text"},{"data":{"items":["**Data Parallelism** — The same operation is applied to different data elements simultaneously. Example: Processing each record in a dataset independently. This is the most common and scalable form of parallelism in distributed systems.","**Task Parallelism** — Different operations (tasks) execute concurrently, potentially on different data. Example: A web request that simultaneously fetches user profile, recommendations, and notifications from different services.","**Pipeline Parallelism** — Work flows through stages, with different stages processing different items concurrently. Example: An ETL pipeline where extraction, transformation, and loading occur in parallel on different batches.","**Bit-Level Parallelism** — Hardware-level parallelism where processors operate on wider data words. Less relevant for application design, but foundational to CPU architecture.","**Instruction-Level Parallelism (ILP)** — CPUs execute multiple instructions simultaneously through pipelining, superscalar execution, and out-of-order execution. Automatic but can be enhanced by cache-friendly code."],"title":{"icon":"Target","text":"Core Parallelism Models","type":"icon","color":"primary","variant":"outline-circle"},"variant":"accent"},"type":"list"},{"data":{"left":[{"data":{"items":["Same code, different data partitions","Scales linearly with data size","Minimal coordination overhead","Embarrassingly parallel workloads","Map operations in MapReduce","Batch processing of events","Image/video frame processing"],"title":{"icon":"CheckCircle","text":"Data Parallelism Characteristics","type":"icon","color":"primary","variant":"outline-circle"},"variant":"accent"},"type":"list"}],"right":[{"data":{"items":["Different code, potentially same data","Scales with number of independent tasks","May require result aggregation","Microservices fan-out patterns","Concurrent API calls","Parallel feature extraction","Background job execution"],"title":{"icon":"Zap","text":"Task Parallelism Characteristics","type":"icon","color":"primary","variant":"outline-circle"},"variant":"accent"},"type":"list"}]},"type":"two-column"},{"data":{"content":"**Granularity of Parallelism:**\n\nThe size of parallelizable work units significantly impacts efficiency:\n\n- **Fine-grained parallelism:** Many small tasks → High coordination overhead, but good load balancing\n- **Coarse-grained parallelism:** Fewer large tasks → Low overhead, but risk of uneven load distribution\n\n**Optimal granularity** balances coordination costs against load distribution. Too fine-grained and you spend more time managing tasks than executing them. Too coarse-grained and some workers sit idle while others are overloaded."},"type":"text"}],"titleType":"number","titleNumber":2},{"id":"parallelism-implementations","title":"Process, Thread, and Coroutine Parallelism","content":[{"data":{"content":"At the implementation level, parallelism can be achieved through different mechanisms, each with distinct tradeoffs in isolation, overhead, and programming complexity."},"type":"text"},{"data":{"rows":[["Processes","Isolated (separate address space)","High (fork/exec)","Expensive (~1-10ms)","CPU-bound, crash isolation"],["Threads","Shared (same address space)","Medium (~10-100µs)","Moderate (~1-10µs)","Mixed CPU/IO, shared state"],["Coroutines/Green Threads","Shared (cooperative)","Very Low (~1µs)","Very Low (~100ns)","I/O-bound, massive concurrency"],["Async/Await","Shared (event loop)","Minimal","Minimal","High-concurrency I/O"]],"caption":"Parallelism Implementation Mechanisms Compared","headers":["Mechanism","Memory","Creation Cost","Context Switch","Best For"]},"type":"table"},{"data":{"content":"$64"},"type":"text"},{"data":{"content":"$65"},"type":"text"},{"data":{"content":"$66"},"type":"text"},{"data":{"icon":"Lightbulb","title":"The Hybrid Approach","content":"Production systems often combine models: multiple processes (for crash isolation and multi-core utilization) each running async I/O (for high concurrency). Example: Gunicorn spawns multiple worker processes, each running an async event loop in frameworks like FastAPI.","variant":"tip"},"type":"callout"}],"titleType":"number","titleNumber":3},{"id":"parallel-patterns","title":"Parallel Patterns in Distributed Systems","content":[{"data":{"content":"At the distributed system level, parallelization takes specific architectural forms. These patterns have been refined through decades of experience at scale."},"type":"text"},{"data":{"items":["**Scatter-Gather (Fan-Out/Fan-In)** — A coordinator distributes work to multiple workers, then aggregates their results. Used in search engines, recommendation systems, and any aggregated query across partitioned data.","**Map-Reduce** — Data is partitioned (map phase), processed in parallel, shuffled by key, then aggregated (reduce phase). Foundation of batch analytics at scale.","**Parallel Pipelines** — Multiple instances of a pipeline run concurrently, each processing a subset of input. Used in stream processing (Kafka partitions + consumer groups).","**Work Stealing** — Idle workers steal tasks from busy workers' queues. Provides automatic load balancing without central coordination.","**Speculative Execution** — The same task runs on multiple workers; the first result wins. Reduces tail latency when some workers are slow (stragglers)."],"title":{"icon":"Target","text":"Essential Distributed Parallelism Patterns","type":"icon","color":"primary","variant":"outline-circle"},"variant":"accent"},"type":"list"},{"data":{"content":"$67"},"type":"text"},{"data":{"fileName":"scatter-gather-example.ts","languages":[{"code":"$68","label":"TypeScript","value":"typescript"}],"showLineNumbers":true},"type":"code"}],"titleType":"number","titleNumber":4},{"id":"synchronization-overhead","title":"Synchronization and Coordination Overhead","content":[{"data":{"content":"Parallelization is not free. Every parallel system pays coordination costs that reduce the net speedup. Understanding and minimizing this overhead is crucial for effective parallelization."},"type":"text"},{"data":{"items":["**Task Creation and Scheduling** — Creating threads/processes, managing task queues, and scheduling work has non-zero cost. For very small tasks, this overhead exceeds the parallelization benefit.","**Context Switching** — When the OS switches between threads/processes, it must save and restore CPU state, flush caches, and update memory mappings. Excessive context switching destroys throughput.","**Lock Contention** — Multiple threads competing for shared locks serialize execution. High-contention locks become the bottleneck, regardless of available parallelism.","**Cache Coherence** — In multi-core systems, changes to shared data must be synchronized across CPU caches (MESI protocol). This causes cache invalidation storms for heavily shared data.","**Memory Bandwidth** — Parallel threads may saturate memory bus bandwidth before exhausting CPU capacity, creating an unexpected bottleneck.","**Network Overhead (Distributed)** — Distributing work across machines adds serialization, network transfer, and coordination costs. Network round-trips (1-100ms) may dwarf computation time."],"title":{"icon":"AlertCircle","text":"Sources of Parallelization Overhead","type":"icon","color":"destructive","variant":"outline-circle"},"variant":"glass"},"type":"list"},{"data":{"rows":[["Create new thread (Java)","~10-100µs","Significant for short tasks"],["Thread context switch","~1-10µs","Multiplies with oversubscription"],["Acquire uncontended lock","~10-100ns","Negligible in isolation"],["Acquire contended lock","~1µs-10ms","Serial bottleneck under contention"],["Cache line invalidation","~10-100ns","Compounds for shared mutable state"],["Network round-trip (same DC)","~0.5-2ms","Dominates for small tasks"],["Network round-trip (cross-region)","~50-200ms","Often prohibitive for fine-grained work"]],"caption":"Approximate Overhead Costs (Order of Magnitude)","headers":["Operation","Approximate Latency","Impact on Parallelization"]},"type":"table"},{"data":{"content":"**Strategies to Minimize Overhead:**\n\n1. **Batch Small Tasks** — Instead of parallelizing 1,000 tasks of 1ms each (overhead-dominated), batch into 10 tasks of 100ms each.\n\n2. **Lock-Free Data Structures** — Use atomic operations and compare-and-swap (CAS) instead of locks where possible. Examples: ConcurrentHashMap in Java, atomic types in C++/Rust.\n\n3. **Thread-Local Storage** — Avoid sharing state by giving each thread its own copy. Merge results only at the end.\n\n4. **Work Stealing Over Work Pushing** — Let idle workers pull work rather than distributing work upfront. This naturally balances load.\n\n5. **Connection Pooling** — For distributed parallelism, reuse network connections rather than establishing new ones per task.\n\n6. **Colocate Data and Compute** — Move computation to data rather than moving data to computation. This is the core insight of MapReduce."},"type":"text"},{"data":{"icon":"Lightbulb","title":"The Goldilocks Principle","content":"Optimal parallelization is about finding the right granularity—fine enough for good load balancing, coarse enough to amortize coordination costs. There's no universal formula; profile your specific workload to find the sweet spot.","variant":"tip"},"type":"callout"}],"titleType":"number","titleNumber":5},{"id":"real-world-examples","title":"Real-World Parallelization Examples","content":[{"data":{"content":"Let's examine how production systems leverage parallelization to achieve massive throughput."},"type":"text"},{"data":{"tabs":[{"id":"databases","label":"Databases","content":[{"data":{"content":"$69"},"type":"text"}],"titleIcon":"Target"},{"id":"web-servers","label":"Web Servers","content":[{"data":{"content":"$6a"},"type":"text"}],"titleIcon":"Zap"},{"id":"data-processing","label":"Data Processing","content":[{"data":{"content":"$6b"},"type":"text"}],"titleIcon":"Target"}]},"type":"tabs"}],"titleType":"number","titleNumber":6},{"id":"antipatterns","title":"Anti-Patterns and Common Pitfalls","content":[{"data":{"content":"Parallelization can backfire when applied incorrectly. These anti-patterns waste resources or actually reduce throughput."},"type":"text"},{"data":{"items":["**Over-Parallelization** — Spawning more threads/processes than CPU cores leads to context switching overhead without benefit. For CPU-bound work, optimal thread count ≈ number of cores.","**Parallelizing Inherently Serial Work** — Some algorithms have dependencies that prevent parallelization. Forcing parallelism on serial work adds overhead without speedup.","**Lock Convoy** — Every thread acquires the same lock in sequence, creating a queue that serializes execution despite parallel threads.","**False Sharing** — Independent data on the same cache line causes unnecessary cache invalidation. Threads writing to adjacent array elements suffer this.","**Unbounded Parallelism** — Spawning unlimited threads for incoming work leads to resource exhaustion, cascading failures, and system crashes under load.","**Thundering Herd** — All workers wake simultaneously for a single event (e.g., accept() on a socket), causing massive contention. Only one succeeds; others wasted cycles.","**Ignoring Backpressure** — Producing work faster than consumers can process leads to unbounded queue growth, memory exhaustion, and crash."],"title":{"icon":"XCircle","text":"Parallelization Anti-Patterns","type":"icon","color":"destructive","variant":"outline-circle"},"variant":"glass"},"type":"list"},{"data":{"icon":"AlertCircle","title":"The Thread Pool Size Formula","content":"For CPU-bound work: threads = CPU cores.\nFor I/O-bound work: threads = CPU cores × (1 + wait time / compute time).\n\nExample: If tasks spend 80% waiting on I/O (wait/compute = 4), optimal threads = cores × 5. Going beyond this trades context switch overhead for diminishing returns.","variant":"error"},"type":"callout"}],"titleType":"number","titleNumber":7},{"id":"summary","title":"Summary: Parallelization for Throughput","content":[{"data":{"content":"We've explored parallelization as the foundational technique for throughput optimization. Let's consolidate the key insights:"},"type":"text"},{"data":{"type":"ordered","items":["**Parallelization multiplies throughput** — By utilizing multiple processing resources simultaneously, systems can handle far more work than sequential execution allows.","**Amdahl's Law sets limits** — Serial portions of a workload cap the maximum speedup. Eliminating serialization is often more impactful than adding more parallelism.","**Choose the right parallelism model** — Processes for isolation, threads for shared state with moderate concurrency, async/coroutines for massive I/O concurrency.","**Distributed patterns enable scale** — Scatter-gather, MapReduce, and work stealing patterns parallelize across machines, not just cores.","**Overhead is the enemy** — Coordination, synchronization, and network costs reduce net gains. Batch work, minimize sharing, and optimize granularity.","**Avoid common pitfalls** — Over-parallelization, lock convoys, and unbounded concurrency can make systems slower, not faster."],"title":{"icon":"CheckCircle","text":"Key Takeaways","type":"icon","color":"primary","variant":"outline-circle"},"variant":"accent"},"type":"list"},{"data":{"content":"**What's next:**\n\nParallelization is just one dimension of throughput optimization. The next page explores **batching**—a complementary technique that amortizes per-operation overhead by grouping multiple operations together, often providing greater throughput gains than parallelization alone for I/O-bound workloads."},"type":"text"},{"data":{"icon":"CheckCircle","title":"Page Complete","content":"You now understand parallelization as a throughput optimization technique—from theoretical foundations (Amdahl's Law) through implementation mechanisms (processes, threads, async) to distributed patterns (scatter-gather, MapReduce) and common pitfalls. Next, we'll examine batching strategies.","variant":"success"},"type":"callout"}],"titleType":"number","titleNumber":8}],"isLocked":false},{"id":"seed_e2cb38c2f8d37539b2628cf2ba99870e","title":"Batching Requests: Amortizing Overhead Through Grouping","order":1,"content":null,"isLocked":true},{"id":"seed_c3af61fa5fef325dcd342e80417fed9f","title":"Connection Reuse: Eliminating Establishment Overhead","order":2,"content":null,"isLocked":true},{"id":"seed_184f130cc9d07f4dfc39495f106094e0","title":"Queue-Based Processing: Decoupling for Throughput","order":3,"content":null,"isLocked":true},{"id":"seed_36076999a3b45394a62154f311878407","title":"Horizontal Scaling: Multiplying Capacity Through Distribution","order":4,"content":null,"isLocked":true}]}},"dataUpdateCount":1,"dataUpdatedAt":1792306257326,"error":null,"errorUpdateCount":0,"errorUpdatedAt":0,"fetchFailureCount":0,"fetchFailureReason":null,"fetchMeta":null,"isInvalidated":false,"status":"success","fetchStatus":"idle"},"queryKey":[["learningContent","getModule"],{"input":{"slug":"throughput-optimization"},"type":"query"}],"queryHash":"[[\"learningContent\",\"getModule\"],{\"input\":{\"slug\":\"throughput-optimization\"},\"type\":\"query\"}]"},{"dehydratedAt":1792306257326,"state":{"data":{"json":null},"dataUpdateCount":1,"dataUpdatedAt":1792306257324,"error":null,"errorUpdateCount":0,"errorUpdatedAt":0,"fetchFailureCount":0,"fetchFailureReason":null,"fetchMeta":null,"isInvalidated":false,"status":"success","fetchStatus":"idle"},"queryKey":[["learningContent","getProgress"],{"input":{"moduleId":"seed_3f1125d5754a425e28726c390c3f3c4d"},"type":"query"}],"queryHash":"[[\"learningContent\",\"getProgress\"],{\"input\":{\"moduleId\":\"seed_3f1125d5754a425e28726c390c3f3c4d\"},\"type\":\"query\"}]"},{"dehydratedAt":1792306257326,"state":{"data":{"json":null},"dataUpdateCount":1,"dataUpdatedAt":1792306257324,"error":null,"errorUpdateCount":0,"errorUpdatedAt":0,"fetchFailureCount":0,"fetchFailureReason":null,"fetchMeta":null,"isInvalidated":false,"status":"success","fetchStatus":"idle"},"queryKey":[["feedback","get"],{"input":{"contentType":"LEARNING_PAGE","contentId":"seed_e2cb38c2f8d37539b2628cf2ba99870e"},"type":"query"}],"queryHash":"[[\"feedback\",\"get\"],{\"input\":{\"contentId\":\"seed_e2cb38c2f8d37539b2628cf2ba99870e\",\"contentType\":\"LEARNING_PAGE\"},\"type\":\"query\"}]"},{"dehydratedAt":1792306257326,"state":{"data":{"json":null},"dataUpdateCount":1,"dataUpdatedAt":1792306257324,"error":null,"errorUpdateCount":0,"errorUpdatedAt":0,"fetchFailureCount":0,"fetchFailureReason":null,"fetchMeta":null,"isInvalidated":false,"status":"success","fetchStatus":"idle"},"queryKey":[["feedback","getUserLike"],{"input":{"contentType":"LEARNING_PAGE","contentId":"seed_e2cb38c2f8d37539b2628cf2ba99870e"},"type":"query"}],"queryHash":"[[\"feedback\",\"getUserLike\"],{\"input\":{\"contentId\":\"seed_e2cb38c2f8d37539b2628cf2ba99870e\",\"contentType\":\"LEARNING_PAGE\"},\"type\":\"query\"}]"},{"dehydratedAt":1792306257326,"state":{"data":{"json":[]},"dataUpdateCount":1,"dataUpdatedAt":1792306257324,"error":null,"errorUpdateCount":0,"errorUpdatedAt":0,"fetchFailureCount":0,"fetchFailureReason":null,"fetchMeta":null,"isInvalidated":false,"status":"success","fetchStatus":"idle"},"queryKey":[["highlights","getHighlights"],{"input":{"contentType":"LEARNING_PAGE","contentId":"seed_e2cb38c2f8d37539b2628cf2ba99870e"},"type":"query"}],"queryHash":"[[\"highlights\",\"getHighlights\"],{\"input\":{\"contentId\":\"seed_e2cb38c2f8d37539b2628cf2ba99870e\",\"contentType\":\"LEARNING_PAGE\"},\"type\":\"query\"}]"},{"dehydratedAt":1792306257326,"state":{"data":"$undefined","dataUpdateCount":0,"dataUpdatedAt":0,"error":null,"errorUpdateCount":0,"errorUpdatedAt":0,"fetchFailureCount":0,"fetchFailureReason":null,"fetchMeta":null,"isInvalidated":false,"status":"pending","fetchStatus":"fetching"},"queryKey":[["highlights","getPopularHighlights"],{"input":{"contentType":"LEARNING_PAGE","contentId":"seed_e2cb38c2f8d37539b2628cf2ba99870e","minCount":2},"type":"query"}],"queryHash":"[[\"highlights\",\"getPopularHighlights\"],{\"input\":{\"contentId\":\"seed_e2cb38c2f8d37539b2628cf2ba99870e\",\"contentType\":\"LEARNING_PAGE\",\"minCount\":2},\"type\":\"query\"}]","promise":"$@6c"}]},"children":"$L6d"}]