Design the Twitter News Feed

To design Twitter's feed, I first clarify that we need sub-second latency for billions of reads while handling massive write spikes. The core challenge is the fan-out mechanism. A pure pull model is too slow for millions of followers, while a pure push model wastes resources on inactive accounts. I propose a hybrid approach: for normal users, we use push to their follower lists upon posting. However, for celebrities with millions of followers, pushing every tweet would overwhelm our infrastructure. Instead, we store their tweets in a dedicated 'hot' cache or a separate read-optimized store, and only pull them during feed generation. This prevents the 'thundering herd' problem. We then aggregate these streams, deduplicate, and pass them through a lightweight ranking service that filters spam and boosts relevant content before serving the final list. Finally, we must ensure durability by writing to a message queue before updating the cache, ensuring no tweets are lost even if the feed service temporarily fails.