Production Twitter on one machine? 100Gbps NICs and NVMe are fast (thume.ca)

Article summary

The article explores the idea of running Twitter on a single machine, discussing the feasibility of handling the platform's load with modern high-performance storage and networking. The author presents a prototype in Rust, demonstrating the potential for serving the full production load with most features intact on a single machine. However, the author acknowledges that this is not a practical or recommended approach, but rather an educational exercise. The article also delves into the math behind Twitter's data storage and retrieval needs.

Main themes

• Systems design
• Scalability
• Compute efficiency
• Twitter architecture
• Real-world complexity
• Theoretical exercises
• Cloud infrastructure costs
• Business requirements vs. technical simplicity

What commenters say

• Some commenters think the title is misleading and the idea of running Twitter on one machine is unrealistic.
• Others argue that the exercise is useful for highlighting potential compute bloat in large companies like Twitter.
• There is disagreement over whether Twitter's high compute usage is necessary or if some features are unnecessary and wasteful.
• Some commenters suggest that Twitter's complexity and requirements, such as analytics and trending, make it difficult to simplify the system.
• Others point out that companies like WhatsApp have achieved high efficiency with relatively low compute resources.
• The discussion touches on the challenges of handling concurrency, notifications, and search on a large scale.
• Some argue that the article ignores real-world functionality and requirements that make Twitter a business rather than a toy.
• There is a question about whether Twitter's high compute usage is justified, given its unprofitability and significant spending on cloud infrastructure.