Designing a URL Shortener

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

URL Storage Calculation
=======================
 
Data per URL entry:
- Short code:          7 bytes (6-8 chars, let's use 7)
- Long URL:          200 bytes (average, can be up to 2KB)
- Creation timestamp:  8 bytes
- Expiration time:     8 bytes (optional)
- User ID:             8 bytes (optional)
- Click count:         8 bytes
- Flags/metadata:     10 bytes
-----------------------------------
Total per URL:       ~250 bytes
 
URLs created per day:        100,000,000
Bytes per day:               100M × 250 = 25 GB/day
Bytes per year:              25 GB × 365 = 9.125 TB/year
Bytes over 10 years:         ~91 TB for URL mappings alone
 
With indexes and replication (3x):
Total storage (10 years):    ~275 TB + indexes ≈ 400 TB

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

Bandwidth Calculation
=====================
 
Redirects per day:           1,000,000,000 (1 billion)
Redirects per second (avg):  1B / 86,400 ≈ 11,574 RPS
Redirects per second (peak): ~50,000 RPS (4-5x average)
 
Request size (incoming):
- HTTP headers:         500 bytes (average)
- Short URL in path:    20 bytes
Total incoming:         ~520 bytes per request
 
Response size (outgoing):
- HTTP 301/302 headers: 300 bytes
- Location header:      200 bytes (destination URL)
- Body (minimal):       100 bytes
Total outgoing:         ~600 bytes per response
 
Incoming bandwidth:
- Average: 11,574 × 520 bytes = 6.0 MB/s
- Peak:    50,000 × 520 bytes = 26 MB/s
 
Outgoing bandwidth:
- Average: 11,574 × 600 bytes = 6.9 MB/s
- Peak:    50,000 × 600 bytes = 30 MB/s
 
Total peak bandwidth: ~60 MB/s (~480 Mbps)
Note: This is modest; network is NOT the bottleneck.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

Cache Sizing Calculation
========================
 
Observation: URL access follows power-law distribution
- 20% of URLs receive 80% of traffic
- Top 1% of URLs receive ~50% of traffic
 
To achieve 90%+ cache hit rate:
- Cache the top 10-20% of URLs by access frequency
 
URLs created per day:    100,000,000
Active URLs (accessed in last 30 days): ~500,000,000
Top 20% active URLs:     100,000,000
 
Memory per cached entry:
- Short code:            7 bytes
- Long URL:            200 bytes
- Metadata (TTL, etc):  50 bytes
- Redis overhead:       50 bytes
Total:                 ~300 bytes
 
Cache size for top 20%:
100M × 300 bytes = 30 GB per region
 
With 5 global regions: 150 GB total cache
(This is very achievable with modern Redis clusters)

1
2
3
4
5
6
7
8
9
10
11
12
13
14

Read/Write Ratio Calculation
=============================
 
Writes (URL creation):      100 million/day = 1,157/second avg
Reads (redirects):        1,000 million/day = 11,574/second avg
 
Read/Write Ratio: 1000M / 100M = 10:1
 
This is actually conservative. In practice:
- Popular links get clicked thousands of times
- Many URLs created by spammers never get clicked
- Marketing campaigns generate millions of clicks per URL
 
Real-world ratio often: 100:1 or higher

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68

// ===============================
// URL Shortener Core API
// ===============================
 
// 1. Create Short URL
// POST /api/v1/shorten
interface CreateShortUrlRequest {
  longUrl: string;              // Required: destination URL
  customAlias?: string;         // Optional: user-requested short code
  expiresAt?: string;           // Optional: ISO timestamp for expiration
  title?: string;               // Optional: descriptive title
  tags?: string[];              // Optional: categorization tags
}
 
interface CreateShortUrlResponse {
  shortCode: string;            // Generated short code (e.g., "a7Xk2B")
  shortUrl: string;             // Full short URL (e.g., "https://short.url/a7Xk2B")
  longUrl: string;              // Original URL (echoed back)
  createdAt: string;            // ISO timestamp
  expiresAt?: string;           // Expiration if set
}
 
// 2. Redirect (Browser-facing, not API)
// GET /{shortCode}
// Response: 301/302 Redirect to longUrl
// Headers: Location: {longUrl}
 
// 3. Get URL Info
// GET /api/v1/urls/{shortCode}
interface GetUrlInfoResponse {
  shortCode: string;
  shortUrl: string;
  longUrl: string;
  createdAt: string;
  expiresAt?: string;
  clicks: number;               // Total click count
  status: "active" | "expired" | "disabled";
}
 
// 4. Get URL Analytics
// GET /api/v1/urls/{shortCode}/analytics
interface GetAnalyticsRequest {
  startDate: string;            // ISO date
  endDate: string;              // ISO date
  granularity: "hour" | "day" | "week" | "month";
}
 
interface GetAnalyticsResponse {
  shortCode: string;
  totalClicks: number;
  uniqueClicks: number;
  clicksByDate: { date: string; clicks: number }[];
  clicksByCountry: { country: string; clicks: number }[];
  clicksByReferrer: { referrer: string; clicks: number }[];
  clicksByDevice: { device: string; clicks: number }[];
}
 
// 5. Update Short URL
// PATCH /api/v1/urls/{shortCode}
interface UpdateUrlRequest {
  longUrl?: string;             // Update destination
  expiresAt?: string;           // Update expiration
  disabled?: boolean;           // Enable/disable
}
 
// 6. Delete Short URL
// DELETE /api/v1/urls/{shortCode}
// Response: 204 No Content