Constraints and Bottlenecks

Every system, no matter how sophisticated, operates within boundaries. The most elegantly architected distributed system, the most cleverly optimized algorithm, the most carefully chosen technology stack—all are ultimately shaped by constraints. These are the immutable facts about your operating environment that define what is possible, what is practical, and what is foolish to attempt.

The difference between a good system architect and a great one often comes down to a single skill: the ability to identify constraints early, understand their implications deeply, and design systems that work harmoniously within these boundaries rather than fighting against them.

In a system design interview or real-world architecture session, one of the most valuable things you can do in the first few minutes is clearly articulate the constraints you're working within. This demonstrates maturity, prevents wasted effort, and sets the foundation for realistic, implementable solutions.

A constraint is any factor that limits your design choices or places bounds on how your system can operate. Constraints are not problems to be solved—they are realities to be acknowledged and worked within. They come in several fundamental categories:

Physical Constraints: These are the laws of physics and the fundamental limitations of computing hardware. The speed of light limits network latency between distant data centers (roughly 1ms per 200km of fiber). Disk seeks take milliseconds. Memory bandwidth is finite. CPU cycles take nanoseconds. These cannot be changed—only worked around.

Resource Constraints: These are the limitations imposed by available resources. You have a finite budget, a specific number of servers, limited storage capacity, and a fixed amount of memory per node. While resources can sometimes be increased, there are always limits—both hard (budget) and soft (diminishing returns).

Operational Constraints: These are the limitations imposed by your organization, processes, and operational capabilities. Your team may only have expertise in certain technologies. Your deployment pipeline may only support certain architectures. Regulatory requirements may mandate specific data handling practices. SLAs may require certain availability guarantees.

Time Constraints: Deadlines and development velocity constraints shape what's possible. A system that takes 18 months to build may be technically superior but unacceptable if the business window is 6 months. Time constraints often force trade-offs between ideal architecture and practical delivery.

The cost of discovering constraints late in the design or development process is devastating. Consider the following scenarios:

Scenario 1: The Late-Discovered Latency Constraint

A team designs an elegant microservices architecture with 15 services, each handling a specific domain. The system is built, tested, and deployed. Then, in production, they discover that the end-to-end latency for a single user request is 2 seconds—because each service call adds 50-100ms of network overhead, and the request path involves 20+ service hops.

The constraint (p99 latency must be under 200ms due to user experience requirements) was never identified early. Now, the entire architecture must be reconsidered.

Scenario 2: The Hidden Storage Constraint

An e-commerce platform stores order history forever, with rich metadata and full audit trails. The data model is designed, the database schema implemented, the application code written. Six months after launch, the database is 5TB and growing rapidly. The team discovers that storage costs are dominating the infrastructure budget, and the original cost projections assumed data would be archived after 12 months—a constraint that was never communicated.

Scenario 3: The Operational Expertise Constraint

An architecture calls for a Kafka-based event streaming platform. It's the right technical choice. But the operations team has never run Kafka in production. There's no monitoring, no runbooks, no institutional knowledge of failure modes. The first production incident takes 12 hours to diagnose because no one understands how to interpret Kafka metrics.

Identifying constraints shouldn't be ad-hoc. Experienced architects follow systematic approaches to ensure nothing is missed. Here's a comprehensive framework for constraint discovery:

Step 1: Enumerate Stakeholders

Constraints come from multiple sources. Product managers know business requirements. Operations teams know deployment limitations. Legal knows regulatory requirements. Finance knows budget limits. Each stakeholder category may hold constraints you'd never discover independently.

Step 2: Ask the Right Questions

For each stakeholder category, there are specific questions that surface constraints:

• Business/Product: What's the budget? What's the timeline? What are the SLA requirements? Are there geographic restrictions? What's the expected growth trajectory?

• Technical/Operations: What infrastructure exists? What technologies does the team know? What monitoring is in place? What are current capacity limits? What's the deployment model?

• Legal/Compliance: Are there data residency requirements? Privacy regulations (GDPR, CCPA)? Audit requirements? Retention policies? Encryption mandates?

• Scale: What's the expected concurrent user count? Data volume? Transaction rate? How much growth is anticipated? What are peak vs. average ratios?

Step 3: Quantify Constraints

Vague constraints are useless. 'The system should be fast' is not a constraint. 'P99 response latency must be under 200ms for API calls' is a constraint you can design to.

For every constraint identified, push for quantification:

• Latency: What percentile? What threshold? Measured from where?
• Availability: What percentage? How is it measured? What's the measurement window?
• Capacity: Exact numbers for users, requests, data volume
• Budget: Specific monthly/annual figures, including both CapEx and OpEx
• Time: Specific dates with clear definitions of 'done'

Step 4: Verify Constraints

Not everything presented as a constraint actually is one. Some 'constraints' are actually preferences, habits, or assumptions that haven't been questioned.

• 'We have to use Oracle' — Is this a true contractual/licensing constraint, or just institutional inertia?
• 'We can't go multi-cloud' — Is this a policy decision that could be changed with the right justification?
• 'The latency must be under 50ms' — Where does this number come from? Is it based on research? User studies? Or arbitrary choice?

Gently probing the 'why' behind stated constraints often reveals flexibility you didn't know you had.

Once constraints are identified and verified, they should actively drive your design decisions rather than being external factors you cope with after the fact. This is a crucial mindset shift.

Example: The Latency-Driven Architecture

Constraint: End-to-end API response time must be under 100ms at p99.

This single constraint immediately eliminates certain architectural patterns and promotes others:

• ❌ Long synchronous call chains across many services
• ❌ Cross-datacenter synchronous calls
• ❌ Real-time aggregation of data from multiple slow sources
• ✅ Caching at multiple layers
• ✅ Precomputation and materialized views
• ✅ Parallel fetching rather than sequential
• ✅ Co-located services with shared memory or local calls

The constraint doesn't limit your design—it shapes it. It provides clarity about which patterns are viable.

Example: The Budget-Driven Architecture

Constraint: Total infrastructure cost must be under $50K/month.

This constraint shapes technology choices:

• ❌ Running everything in memory for speed
• ❌ N+2 redundancy for every component
• ❌ Premium database licensing
• ✅ Strategic caching with TTL expiration
• ✅ Right-sized redundancy based on component criticality
• ✅ Open-source alternatives where appropriate
• ✅ Efficient resource utilization over pure performance

In any given design scenario, multiple constraints exist, but typically one or two are binding—meaning they are the tightest limits that constrain the solution space more than any others. Identifying the binding constraint is crucial because it's where optimization effort has the highest return.

The Theory of Constraints in System Design

Borrowed from manufacturing, the Theory of Constraints (TOC) teaches that every system has a bottleneck—a single point that limits the throughput of the entire system. Improving any non-binding constraint provides no benefit to overall throughput; only improving the binding constraint moves the needle.

Example: Finding the Binding Constraint

Consider a system with these constraints:

• Budget: $100K/month
• Latency: p99 under 500ms
• Availability: 99.9%
• Throughput: 10K requests/second
• Team: 5 engineers

For a straightforward web application, the latency and throughput requirements are easily achievable with modern infrastructure. 99.9% availability requires some redundancy but isn't exotic. The $100K budget is generous for this scale.

But the team size—5 engineers—is the binding constraint. With only 5 engineers, you cannot build and operate 30 microservices, each with their own deployment pipelines, monitoring dashboards, and on-call rotations. The team constraint shapes the architecture toward fewer, larger services, simpler operational patterns, and heavy reliance on managed services.

Change the team to 50 engineers, and the binding constraint shifts. Now, with ample engineering capacity, other constraints (perhaps latency or availability for a more demanding use case) become binding instead.

Dynamic Binding Constraints

Binding constraints are not static. As your system evolves, as your team grows, as requirements change, different constraints become binding. A startup might be team-constrained initially, then budget-constrained, then scale-constrained as success arrives.

Smart architects regularly re-evaluate which constraint is binding and adjust architectural direction accordingly. Yesterday's optimal design may not be optimal when the constraint landscape shifts.

In system design interviews, explicitly asking for and clarifying constraints is one of the strongest signals of senior engineering capability. It demonstrates that you think like an architect, not just a builder.

What Interviewers Look For:

When you ask about constraints, you're showing:

1. Maturity: You understand that good design is constrained design
2. Experience: You've encountered real-world constraints that shaped your past work
3. Pragmatism: You're focused on buildable solutions, not theoretical perfection
4. Communication: You know how to gather information collaboratively

Constraint-Focused Interview Opening:

After the problem is stated, your first 5-10 minutes should include questions like:

• 'What's the expected scale? Users, requests per second, data volume?'
• 'What are the latency requirements? Are there specific SLAs?'
• 'What's the availability requirement? Can we tolerate any downtime?'
• 'Are there any budget constraints I should be aware of?'
• 'Is this greenfield, or are there existing systems to integrate with?'
• 'Are there any compliance or regulatory requirements?'

Using Constraints to Structure Your Design:

Once constraints are established, reference them throughout your design:

• 'Given the 99.99% availability requirement, we'll need active-active replication across at least two regions...'
• 'Since we're targeting sub-50ms p99 latency, I'm going to introduce a caching layer here...'
• 'With a 10-person team, I'd suggest a modular monolith rather than microservices to reduce operational complexity...'

This continuous reference to constraints shows you're designing deliberately within boundaries, not just throwing architecture at the wall.

Let's examine how constraints shaped the designs of well-known systems:

Example 1: Google Search

Binding Constraint: Latency

Google discovered early that search latency directly correlates with user engagement and ad revenue. Studies showed that adding 100ms of latency reduced search queries by measurable percentages. This latency constraint drove massive architectural decisions:

• Serving infrastructure distributed globally to minimize network round trips
• Aggressive caching and precomputation
• Tight budgets for every component—each service has millisecond-level latency budgets
• Specialized hardware and networking to shave microseconds

Example 2: Netflix

Binding Constraint: Availability

Netflix operates in a domain where users have zero tolerance for outages—if Netflix is down, users simply go elsewhere. This availability constraint shaped their famous Chaos Engineering culture:

• Active-active multi-region deployment from the start
• Chaos Monkey randomly terminating instances to ensure resilience
• Circuit breakers and fallbacks throughout the system
• Eventual consistency accepted as the trade-off for high availability

Example 3: WhatsApp

Binding Constraint: Team/Operational Simplicity

WhatsApp famously scaled to hundreds of millions of users with a remarkably small engineering team. The team size constraint shaped every decision:

• Erlang for actor-model concurrency with minimal operational overhead
• Extreme focus on operational simplicity
• Monolithic-ish architecture rather than microservices
• Heavy use of standard, well-understood protocols

The team constraint forced elegance. Without the option to throw more engineers at the problem, WhatsApp had to build systems that could scale without proportional team growth.

Constraint identification is not merely a preliminary step in system design—it is the foundation upon which all subsequent decisions rest. Let's consolidate the key insights from this page:

What's Next:

With a strong foundation in constraint identification, we'll now examine the specific types of bottlenecks that commonly constrain system performance and scalability. The next page explores CPU, memory, network, and disk bottlenecks—the fundamental resource limitations that every system architect must understand.