URL Shortener - Step 4: Refinement for Scale
Step 4 of 6: R - Refinement for Scale
Address non-functional requirements with scaling strategies
🧮 Detailed Scale Calculations
📈 Queries Per Second
Write Operations:
100M URLs/day ÷ 86,400s = 1,200 QPS
Peak (3x): 3,600 write QPS
Read Operations (10:1):
1B redirects/day ÷ 86,400s = 11,600 QPS
Peak (3x): 34,800 read QPS
💾 Storage Requirements
Per URL:
short_code: 7 bytes
long_url: ~500 bytes
metadata: ~200 bytes
Total: ~700 bytes/URL
10 Year Total:
365B URLs × 700 bytes = 256 TB
With 3x replication: 768 TB
🖥️ Infrastructure Sizing
Write Servers
3,600 QPS ÷ 1000 QPS/server
= 4 servers (+ 2 for HA)
Read Servers
34,800 QPS ÷ 1000 QPS/server
= 35 servers (+ 15 for HA)
Cache Memory
20% hot URLs = 73B URLs
73B × 700B = 51 TB Redis
🚀 Scaled Architecture
🔀 Database Sharding Strategy
Shard by Short Code
shard_id = hash(short_code) % 32
- • Even distribution across shards
- • O(1) shard lookup
- • Easy to add more shards
- • No hotspots with good hash
Consistent Hashing
Virtual nodes: 150 per physical
- • Minimal data movement on scale
- • ~1/N keys move (N = nodes)
- • Handles node failures gracefully
- • Auto-rebalancing
💾 Multi-Layer Caching
L1: CDN Cache
Edge locations worldwide
• TTL: 1 hour
• Hit rate: ~30%
• Reduces latency to <10ms
L2: Redis Cache
Application-level cache
• TTL: 24 hours
• Hit rate: ~60%
• LRU eviction
L3: DB Cache
Database buffer pool
• In-memory pages
• Hit rate: ~90%
• Automatic management
💡 Scale Refinements Complete
Infrastructure
50 read servers, 6 write servers, multi-region deployment
Data Distribution
32 shards with consistent hashing, 3x replication
Performance
3-layer caching, CDN edge, async analytics