Things I learned from building a production database (maheshba.bitbucket.io)

Article summary

The article shares 42 lessons learned from building a production database, Delos, at Facebook. The author, a systems researcher, led a team that built Delos from scratch and scaled it to a large team without experiencing severe outages. The lessons cover various aspects of building and maintaining a production database, including design, implementation, testing, and collaboration with customers and team members. The article aims to help others who are building new infrastructure at large companies.

Main themes

• building production databases
• team collaboration and communication
• project management and scaling
• industry terminology and culture
• measurement and metrics
• technical design and implementation
• customer interaction and feedback

What commenters say

• Some commenters found the list of lessons to be excellent and relevant to their own experiences, while others had not heard of the term 'Individual Contributor' before and found it confusing.
• The 'kill-it' phase of a project, where everyone tries to kill it, is a valuable phase that allows a small team to create something coherent before it is scaled up.
• Interfacing directly with customer ICs is necessary to understand their concerns and avoid misunderstandings, but it should be done with the oversight of the primary point of contact.
• The term 'Individual Contributor' is commonly used in the tech industry to refer to a person who is not a manager, but its usage may vary across companies and regions.
• Starting with a small skilled team and planning for a larger team is critical for project success, especially with complex projects like building a production database.
• Measurement and metrics should be used as a means to an end, not an end in itself, to avoid losing sight of the project's purpose.
• The article's lessons can be applied to a larger audience beyond those building new infrastructure at large companies, and some principles, such as avoiding over-reliance on metrics, are universally relevant.
• The use of acronyms like 'IC' can be alienating to those who are not familiar with them, and authors should strive to define them clearly to avoid confusion.