Rust std fs slower than Python? No, it's hardware (xuanwo.io)

Article summary

The article investigates a performance difference between Rust and Python when reading files, initially suggesting that Rust's std fs is slower than Python. However, after a thorough analysis using strace and benchmarking, the author discovers that the issue is not with Rust or its standard library, but rather with the hardware, specifically an AMD CPU bug. The author also experiments with using jemalloc, which improves the performance of the Rust program. The article concludes that the performance difference is due to the CPU bug and not a software issue.

Main themes

• Rust vs Python performance
• Hardware bugs and quirks
• Memory allocation and jemalloc
• Language implementation and optimization
• Interpreted languages and performance
• Python object model and ABI

What commenters say

• The article's investigation is a great example of thorough debugging and performance analysis.
• The performance difference between Rust and Python is not due to the languages themselves, but rather the underlying hardware and its quirks.
• The use of jemalloc can significantly improve the performance of Rust programs, especially when dealing with large memory allocations.
• The concept of 'interpreted language' is not well-defined and can be misleading, as it depends on the implementation and not the language itself.
• Python's dictionary implementation is highly optimized and can outperform C++'s std::map in certain scenarios.
• The Python object model can be a limiting factor in terms of performance, but it is also a key part of the language's ABI and ecosystem.