Design a Twitter Feed (Conceptual Data Storage)

To design a Twitter feed capable of handling billions of posts, I would prioritize the Fan-Out-On-Write model to ensure low-latency reads. When a user posts, we push that content ID into the pre-computed timeline lists of all their followers. We store these timelines in Redis because it offers extremely fast retrieval times essential for a scrollable feed. For the actual tweet content, we would persist it in a columnar store like Cassandra, which scales horizontally well for massive write volumes. However, pushing to everyone fails for celebrities with millions of followers. For these accounts, we switch to a hybrid approach where we only push to a subset or use fan-out-on-read, fetching recent tweets dynamically and merging them with the cached list. This prevents system overload during viral events. We also need to handle pagination efficiently. Since Redis lists can grow large, we might split timelines into shards based on time windows, allowing us to fetch specific ranges without loading the entire history. Finally, we must ensure eventual consistency; if a user follows someone and doesn't see the immediate last post, it is acceptable as long as the data converges quickly. This architecture balances the heavy write load of posting with the critical read performance needed for user experience.