How Discord stores trillions of messages (2023) (discord.com)

Article summary

Discord migrated its message storage from Cassandra to ScyllaDB, a Cassandra-compatible database, to improve performance and reduce latency. The migration was necessary due to Cassandra's high maintenance needs and performance issues, despite its scalability. Discord implemented data services to reduce traffic to the database and improve performance. The migration resulted in improved latency and reduced the number of nodes required.

Main themes

• Database migration
• Scalability and performance
• Cassandra vs ScyllaDB
• Distributed systems
• Cloud architecture

What commenters say

• Working at scale can be overwhelming and frustrating, with many engineers expressing relief that they don't work on systems of this size.
• Some engineers prefer working on smaller-scale systems, such as embedded systems, due to their perceived simplicity and lack of bureaucracy.
• The use of new and trendy technologies can be driven by blog posts and leadership pressure, rather than actual problem-solving needs.
• The article's description of Discord's architecture is surprisingly simple, leading some to wonder if modern cloud architecture is often over-engineered.
• The choice of database and architecture depends on specific use cases and requirements, with no one-size-fits-all solution.
• Some engineers appreciate the complexity and challenge of working on large-scale systems, finding it more engaging and creative than smaller-scale work.