The art of high performance computing (theartofhpc.com)

Article summary

The article discusses the art of high-performance computing (HPC) and introduces a series of textbooks by Victor Eijkhout, a research scientist at the Texas Advanced Computing Center. The textbooks cover topics such as computer architecture, parallel programming, and scientific programming. They aim to provide a comprehensive introduction to HPC and are designed for students and practitioners in the field. The textbooks are available for free and can be used as a resource for learning HPC.

Main themes

• HPC education and resources
• hardware architecture and optimization
• HPC software ecosystem
• domain knowledge and technical skills
• scalability and performance
• research and development in HPC

What commenters say

• HPC practitioners need to have a good understanding of hardware architecture to optimize performance, but abstractions and tooling can also be sufficient in many cases.
• The use of shell scripts and batch jobs in HPC is a necessary evil, but can be cumbersome and limiting.
• Domain knowledge is more valuable than purely technical skills in HPC, and researchers often prioritize scientific questions over IT details.
• The HPC software ecosystem has arguably regressed in recent years, with outdated tools and proprietary systems still in widespread use.
• Bayesian optimization can be an effective method for optimizing performance in HPC, especially when dealing with complex parameter spaces.
• Scalability is a key concern in HPC, and optimizing for performance is secondary to ensuring that systems can scale effectively.
• The separation of roles between researchers and developers can be beneficial in HPC, with researchers focusing on scientific questions and developers handling software development and optimization.