Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache (blog.cloudflare.com)

Article summary

Cloudflare optimized the DNS cache of their 1.1.1.1 service, reducing memory usage by 100 terabytes. The optimization involved changing how cache entries are stored in memory, including replacing Vec with Box and reducing the number of lists and pointers. This change also improved the cache's performance, with a 43% increase in insert throughput and a 19% decrease in lookup latency. The optimization was achieved through a series of five successive changes to the cache's data structure.

Main themes

• DNS cache optimization
• premature optimization
• data structure choice
• performance measurement
• programming language trade-offs
• system programming
• scalability
• optimization strategies

What commenters say

• Premature optimization can be a waste of time and resources, and it's better to focus on getting the system working first and optimizing later.
• Using the right data structures from the beginning can avoid future optimization needs and migration pains.
• The decision to optimize or not depends on the specific constraints and priorities of the project, and what is considered a trivial optimization can become important at scale.
• Measuring performance and identifying bottlenecks is crucial before optimizing, and optimizing without measurement can be unnecessary and even harmful.
• The choice of programming language and its features can impact the ability to optimize and use certain techniques, such as arenas in Rust.
• Optimization is not always a linear process, and fighting the tightest bottleneck may not always be the best approach.
• System programming and understanding of low-level details are still important for achieving high performance and efficiency.
• The importance of optimization depends on the scale and specific requirements of the system, and what is cheap and negligible at small scale can become significant at large scale.