How fast are Linux pipes anyway? (2022) (mazzo.li)

Article summary

The article explores the performance of Linux pipes by optimizing a test program that writes and reads data through a pipe, achieving a twentyfold improvement in performance. The optimization process involves understanding how pipes are implemented internally, using Linux's perf tooling, and applying various techniques such as using vmsplice and splice syscalls, huge pages, and busy looping. The article also discusses the limitations and bottlenecks of pipes, including the overhead of copying data between user and kernel space. The optimized program achieves a throughput of around 35GiB/s, comparable to the performance of a highly optimized FizzBuzz program.

Main themes

• Linux pipes performance
• Inter-process communication
• Concurrency and parallelism
• Kernel internals
• Optimization techniques
• Programming language design
• System architecture
• Message passing and synchronization
• Deadlocks and determinism

What commenters say

• Pipes can exhibit non-deterministic behavior due to concurrent execution of commands, which can lead to unexpected output.
• The use of pipes can be less efficient than other methods of inter-process communication, such as shared memory or message passing.
• Some programming languages and shells, like Powershell, implement pipelines deterministically, avoiding the non-determinism seen in Linux pipes.
• The performance of pipes is limited by the speed of the kernel's memory mapping and copying operations, which can be a bottleneck for high-performance applications.
• The use of threads can be less important than message passing and pipes for building concurrent systems, as pipes can provide a cleaner and more efficient way to structure programs.
• Message passing can also lead to deadlocks if not implemented carefully, and is not inherently superior to threads or other synchronization mechanisms.
• The performance of pipes can be affected by the underlying CPU architecture, including the size and organization of caches and the memory controller.
• Optimizing pipe performance requires a deep understanding of Linux kernel internals and the use of specialized tools and techniques, such as perf and vmsplice.
• Pipes can still be a useful and efficient way to structure programs, even if they are not perfectly deterministic or optimized for performance.
• The choice between using pipes, threads, or other synchronization mechanisms depends on the specific requirements and constraints of the application or system being built.