We found a bug in Go's ARM64 compiler (blog.cloudflare.com)

Article summary

Cloudflare discovered a bug in Go's arm64 compiler that causes a race condition in the generated code, leading to rare but frequent panics on arm64 machines. The bug was found through monitoring and investigation of sporadic panics in a service that configures the kernel to handle traffic. The root cause was determined to be a non-atomic adjustment of the stack pointer, which was fixed by using a temporary register to hold the full value being added to the stack pointer. The bug was likely only noticeable due to Cloudflare's large scale.

Main themes

• Compiler bugs
• Go programming language
• arm64 architecture
• Cloudflare infrastructure
• Debugging and investigation
• Assembly language

What commenters say

• The bug was likely only noticeable due to Cloudflare's large scale, and may not have been discovered otherwise.
• The use of assembly language in Go allows for direct access to low-level instructions, which can lead to bugs like this one.
• The fix for the bug involved using a temporary register to hold the full value being added to the stack pointer, which ensures atomicity.
• Some commenters suggested that the assembler could have been modified to handle large constants in a special way, but this was not the chosen solution.
• The discussion highlights the importance of understanding low-level details of computer architecture and assembly language, even for high-level programming languages like Go.
• Some commenters noted that compiler bugs are not uncommon, but are often only discovered at large scales or in specific use cases.
• The article and discussion demonstrate the value of careful debugging and investigation in identifying and fixing complex technical issues.