news.volyx.in

Jemalloc Postmortem (jasone.github.io)

801 points by jasone · 413 days ago · 233 comments on HN

Article summary

The jemalloc memory allocator, first conceived in 2004, has reached the end of its active upstream development. The article describes the development phases of jemalloc, from its inception to its integration with FreeBSD, Firefox, and Facebook. The author reflects on the successes and failures of the project, including the challenges of maintaining a complex codebase and the importance of telemetry data in informing development decisions. The project's conclusion is attributed to a shift in Facebook's priorities, which no longer align with the needs of external users.

Main themes

  • memory allocation
  • software maintenance
  • open-source development
  • performance optimization
  • technology evolution
  • community engagement

What commenters say

  • Some argue that software development can be largely complete, with only bug fixes and security updates needed, while others believe that continuous updates are necessary to keep pace with changing technology.
  • The performance impact of switching to a new memory allocator may be positive, but it depends on the specific use case and allocation profile.
  • Jemalloc's maintenance has been criticized for being too focused on Facebook's internal needs, rather than general utility, leading to a decline in its usefulness for external users.
  • Alternative memory allocators, such as mimalloc and tcmalloc, are being considered as potential replacements for jemalloc, with some users reporting positive experiences with these alternatives.
  • The lack of awareness about external uses and needs has led to decisions that negatively impacted some users, such as the removal of Valgrind support.
  • Some believe that large-scale profiling data is essential for developing high-performance memory allocators, while others think that it is possible to develop effective allocators through other means.
  • The integration of jemalloc with specific projects, such as Redis and Ruby on Rails, may make it difficult to switch to alternative allocators, despite potential benefits.