The computers are fast, but you don't know it (shvbsle.in)

Article summary

The article discusses the author's experience optimizing a Python function for a machine learning service, which initially took 8 seconds to perform 1000 calls and was optimized to take less than 500ms. The author achieved this through several optimizations, including replacing Pandas with simple Python lists, using Cython, and writing the function in pure C++. The optimizations resulted in significant speed improvements, with the final version taking approximately 6.59 seconds for 1 million calls. The author notes that understanding how fast computers can be is important for optimizing code.

Main themes

• optimization techniques
• Pandas vs Python lists
• Cython and C++
• performance trade-offs
• dynamic scripting languages
• concurrent programming
• network latency
• software development costs

What commenters say

• Some commenters argue that the author's optimization was likely due to preallocating weights and not solely because of replacing Pandas with Python lists.
• Others suggest that Pandas is not inherently slow, but rather its usage can be optimized for better performance.
• A few commenters mention that dynamic scripting languages like Python can lead to slower code and that understanding how fast computers can be is crucial for optimization.
• Some argue that the cost of optimizing code is often outweighed by the benefits of saved time and resources, while others claim that it is cheaper for companies to waste CPU cycles on bad performance than to hire engineers to optimize code.
• There is a disagreement on whether bad performance is an externality that affects everyone or just the customer, with some arguing that it has a cumulative opportunity cost on civilization.
• A few commenters note that network latency can be a significant factor in slow performance, but it can be worked around with concurrent programming techniques.
• Some argue that the use of high-level languages can lead to slower code, but others claim that modern mid-level statically typed languages can offer a sweet spot for productivity and performance.