Designing Twitter/X: Social Media Feed at Scale

When you open Twitter (now X), your timeline loads in milliseconds—a personalized stream of tweets from the hundreds or thousands of accounts you follow. This seemingly simple interaction masks one of the most challenging distributed systems problems in modern software engineering.

Consider the scale:

• Over 500 million tweets are posted daily
• The platform serves 350+ million monthly active users
• Average user follows 400+ accounts
• Celebrity accounts have 100+ million followers
• Timeline must load in under 200ms globally

Designing a system that can handle this scale while maintaining sub-second response times requires deep understanding of distributed systems, data modeling, caching strategies, and algorithmic trade-offs. In this module, we'll design Twitter's core features from scratch.

Before diving into technical requirements, we must deeply understand what Twitter/X actually does. This understanding guides every technical decision we make.

Twitter's Core Value Proposition:

Twitter is a real-time information network where users share short-form content with followers. Unlike Facebook's bidirectional friendships, Twitter uses a unidirectional follow model—you can follow anyone without their approval. This asymmetric relationship creates unique technical challenges:

• Power law distribution: A small number of accounts (celebrities, influencers) have millions of followers, while most accounts have few
• Information flows one way: When Taylor Swift tweets, 90 million people receive it; when a regular user tweets, perhaps 200 people see it
• Real-time expectations: Breaking news, live events, and conversations demand near-instant delivery

The Unidirectional Follow Model Implications:

This seemingly simple design choice has profound technical implications:

1. Asymmetric data access patterns: Reading a celebrity's tweets is common; reading a regular user's tweets is rare
2. Fan-out explosion: A single tweet from a popular account triggers millions of deliveries
3. Hot partition risks: Celebrity accounts become data hotspots
4. Staleness tolerance varies: Users following 50 accounts expect perfect chronology; users following 5,000 accept some delay

Functional requirements define what the system must do. For Twitter, we'll focus on three core features: posting tweets, following users, and viewing timelines. Each has nuanced sub-requirements that significantly impact design.

2.1 Posting Tweets (Write Path)

The tweet posting feature is deceptively complex. Let's decompose it:

2.2 Following Users (Relationship Management)

The follow system establishes the social graph that powers content distribution:

2.3 Viewing Timeline (Read Path)

The timeline is the most complex and performance-critical feature:

Non-functional requirements (NFRs) define how well the system must perform. For Twitter, these NFRs drive architectural decisions more than functional requirements do.

3.1 Scalability Requirements

3.2 Availability Requirements

Twitter is a critical communication platform, especially during emergencies and breaking news:

3.3 Latency Requirements

User experience depends critically on response times:

3.4 Consistency Requirements

Twitter can tolerate some consistency trade-offs in favor of availability and performance:

Before designing any system, we must estimate the scale. These calculations inform technology choices, capacity planning, and architecture patterns.

4.1 Traffic Estimation

4.2 Storage Estimation

4.3 Bandwidth Estimation

Every system has constraints that shape its design. For Twitter, several constraints are particularly critical:

5.1 The Celebrity Problem (Hot Partitions)

The most challenging constraint in Twitter's design is the power-law distribution of followers:

This creates a fundamental design challenge: should we pre-compute timelines (fanout on write) or compute them on request (fanout on read)? We'll explore both approaches in detail later.

5.2 Real-Time Expectations

Users expect tweets to appear in their timeline within seconds of posting. For breaking news and live events, this expectation is even higher. This constraint impacts:

• Message queue depth and throughput
• Cache invalidation strategies
• WebSocket/SSE connection management
• Global data replication delays

5.3 Viral Content (Thundering Herd)

When content goes viral, millions of users may request the same tweet simultaneously. Without careful caching:

• Database becomes bottleneck
• Cache stampedes exhaust memory
• Response times degrade exponentially

Before designing internal systems, let's define the external API contract. This clarifies exactly what the system must support.

6.1 Tweet APIs

6.2 Follow APIs

6.3 Timeline APIs

Before building, we must define how we'll measure success. Service Level Indicators (SLIs) and Service Level Objectives (SLOs) provide concrete targets.

7.1 Key Performance Indicators

7.2 Business Metrics to Track

We've completed a thorough requirements analysis for our Twitter-like system. Let's consolidate our understanding:

What's Next:

With requirements established, we'll explore how to actually build this system. The next page dives into Feed Generation Approaches—the architectural patterns for assembling a user's timeline from hundreds of followed accounts efficiently. We'll examine pull-based, push-based, and hybrid approaches, each with distinct trade-offs.

The requirements we've defined here will directly inform which approach works best for different user segments. The celebrity problem, in particular, will drive us toward a hybrid architecture that treats different account types differently.