Non-Functional Requirements

Every ambitious system eventually confronts the same question: What happens when success arrives? A startup's prototype handling 100 users per day suddenly faces 100,000. A regional service expanding nationally sees traffic multiply by 50x overnight. An API serving a handful of partners becomes the backbone for an entire ecosystem.

Scalability requirements define the answer to this question before it becomes a crisis. They are not merely technical specifications—they are strategic decisions that determine whether your system will gracefully accommodate growth or collapse under its own success.

In system design interviews and real-world architecture, the ability to precisely articulate scalability requirements separates engineers who build systems that survive from those who build systems that thrive.

Scalability is one of the most frequently invoked yet poorly understood concepts in system design. Many engineers equate scalability with "handling more users," but this oversimplification masks the nuanced reality of what scalability actually means.

The Formal Definition:

Scalability is a system's ability to maintain or improve its performance characteristics (throughput, latency, reliability) as the demand placed upon it increases across one or more dimensions, typically by adding resources proportionally.

This definition reveals three critical aspects often overlooked:

1. Performance characteristics must be maintained — A system that handles 10x users but with 100x latency hasn't scaled; it has degraded.
2. Multiple dimensions exist — Users, data volume, geographic spread, and feature complexity all constitute different scaling axes.
3. Resource proportionality matters — Scaling that requires exponentially more resources than workload growth is economically unscalable.

Every scalability decision fundamentally reduces to choosing between two strategies—or more precisely, determining the right blend of both. Understanding the deep implications of vertical and horizontal scaling is essential for defining appropriate scalability requirements.

Vertical Scaling (Scaling Up):

Vertical scaling increases system capacity by adding more resources to existing nodes—more CPU cores, more RAM, faster storage, enhanced network bandwidth. This approach is conceptually simple: when capacity is exhausted, upgrade the hardware.

Horizontal Scaling (Scaling Out):

Horizontal scaling increases capacity by adding more nodes to the system. Instead of making one server more powerful, you add more servers and distribute the workload across them.

The Strategic Decision Framework:

When specifying scalability requirements, you must indicate which scaling strategy your design assumes or mandates:

The Hybrid Reality:

In practice, most scalable systems employ both strategies. Databases often scale vertically first (larger instances) before sharding horizontally. Compute tiers typically scale horizontally (stateless web servers) while specialized components scale vertically (in-memory caches).

Your scalability requirements should acknowledge this complexity: "The system shall scale horizontally for stateless compute workloads while supporting vertical scaling for database read replicas up to [instance size], with horizontal sharding available when vertical limits are reached."

Vague scalability requirements like "the system should be scalable" or "handle high traffic" are meaningless for architectural decisions. Precise quantification transforms aspirations into actionable specifications.

The Three-Horizon Framework:

Scalability requirements should be defined across three time horizons:

1. Current State (Day 1) — What the system must handle at launch
2. Near-Term Growth (Year 1) — Expected load after predictable growth
3. Long-Term Target (Year 3-5) — Aspirational scale informing architectural choices

This framework prevents both over-engineering (building for 100M users when you have 100) and under-engineering (re-architecting every 6 months).

Deriving Metrics from Business Requirements:

Scalability metrics don't appear from thin air—they flow from business context. Here's a structured approach to deriving them:

Step 1: Identify User Archetypes

• How many users will the system serve?
• What's the growth trajectory? (Linear, exponential, step-function at launches)
• What's the geographic distribution?

Step 2: Map User Behavior to System Load

• How many actions per active user per session?
• What's the session duration and frequency?
• What percentage of actions are reads vs. writes?

Step 3: Apply Traffic Pattern Analysis

• What's the peak-to-average ratio? (Typically 3-10x for consumer apps)
• Are there predictable spikes? (Black Friday, campaign launches)
• What's the growth rate month-over-month?

Step 4: Calculate Derived Metrics

• Peak RPS = (DAU × Actions per User × Peak Multiplier) ÷ Active Hours in Seconds
• Storage Growth = New Users × Data per User + Existing Users × Data Accumulation Rate
• Connection Pool = Peak Concurrent Users × Connections per User Session

Scalability requirements must go beyond stating target capacities—they should define when and how scaling actions occur. This transforms scalability from a static property into a dynamic capability.

Elastic Scaling Specifications:

Modern systems don't have fixed capacity; they adjust dynamically based on demand. Your requirements should specify the elasticity characteristics:

1. Scale-Up Trigger — What conditions initiate capacity increase?
2. Scale-Up Behavior — How much capacity is added and how quickly?
3. Scale-Down Trigger — When can capacity be safely reduced?
4. Scale-Down Behavior — How is capacity reduced without disruption?
5. Minimum Capacity — What baseline must always be maintained?
6. Maximum Capacity — What ceiling protects against runaway costs or cascading failures?

Metric Selection for Scaling Triggers:

The metrics you choose for scaling triggers dramatically affect system behavior:

Pre-Emptive vs. Reactive Scaling:

Advanced scalability requirements distinguish between reactive scaling (responding to current load) and pre-emptive scaling (anticipating future load):

Reactive Scaling Specification: "The system shall automatically scale compute capacity in response to observed load, maintaining target utilization between 40-70% with actions triggered within 60 seconds of threshold breach."

Pre-Emptive Scaling Specification: "The system shall pre-scale capacity 30 minutes before scheduled events (marketing campaigns, product launches per schedule feed), achieving target capacity before the first anticipated traffic increase."

Predictive Scaling Specification: "The system shall utilize historical traffic patterns to predictively scale capacity, targeting 20% headroom above the 7-day moving average for the corresponding time window."

No system scales infinitely. Understanding and documenting scalability constraints is as important as specifying targets. These constraints drive architectural trade-offs and inform capacity planning.

Universal System Laws:

Two fundamental laws govern system scalability:

Amdahl's Law:

Speedup from parallelization is limited by the sequential portion of the workload. If 20% of your processing must happen sequentially (e.g., database locks, ordered operations), maximum speedup is 5x regardless of how many processors you add.

Implication: Identify sequential bottlenecks in your requirements: "System scalability is constrained by sequential order processing, which limits throughput to 10K orders/second regardless of compute scale."

Universal Scalability Law (USL):

USL extends Amdahl's Law to account for contention (coherency delays) in distributed systems. As you add nodes, communication overhead can actually cause negative scaling—more instances means lower throughput.

Implication: "Cross-instance coordination for distributed transactions introduces non-linear overhead. Beyond 50 instances, throughput per instance degrades by approximately 5% per additional instance."

Scalability requirements are meaningless without corresponding testing requirements. How will you verify your system actually meets its scalability specifications?

Categories of Scalability Testing:

Scalability Testing Specification Template:

Your scalability requirements should include testing specifications:

Scalability Testing Requirements:

1. Load Test Baseline
   - Frequency: Weekly automated, daily in pre-prod
   - Target: 100% of Day 1 specification (5K RPS)
   - Duration: 30 minutes sustained
   - Pass criteria: P99 latency < 200ms, error rate < 0.1%

2. Stress Test Maximum Capacity
   - Frequency: Monthly
   - Target: Increase until 1% error rate sustained
   - Documentation: Record breaking point capacity
   - Minimum acceptable: 150% of Year 1 target (15K RPS)

3. Spike Test Elasticity
   - Frequency: After any scaling configuration change
   - Pattern: 0% → 100% → 300% → 100% → 0% over 30 minutes
   - Pass criteria: P99 recovers within 5 minutes of scale event
   - Auto-scaling: Verify instances added within SLA

4. Soak Test Stability
   - Frequency: Weekly, 24-hour duration
   - Target: 70% of maximum capacity
   - Pass criteria: No memory growth > 10%, no latency degradation > 20%

With all the concepts covered, let's synthesize them into a framework for writing complete, actionable scalability requirements.

The SMART-S Framework for Scalability:

• Specific — Precise metrics, not vague aspirations
• Measurable — Clear quantitative targets with units
• Achievable — Realistic given technology and timeline
• Relevant — Tied to business needs and user experience
• Time-bound — Specified for defined time horizons
• Scalable — Explicitly addresses growth trajectory

We have covered the complete landscape of scalability requirements. Let's consolidate the essential takeaways:

What's Next:

With scalability requirements mastered, we turn to the next critical non-functional requirement: Availability. While scalability determines whether your system can handle growth, availability determines whether your system is accessible when users need it. These two requirements often create tension—optimizing for one can compromise the other—making it essential to understand both deeply.