OpenAI unveils its first custom chip, built by Broadcom (techcrunch.com)

Article summary

OpenAI has unveiled its first custom-built inference processor, named Jalapeño, designed and manufactured in collaboration with Broadcom. The new processor is specifically designed for the unique needs of OpenAI's inference systems and has shown significantly better performance-per-watt than current state-of-the-art alternatives. The partnership aims to reduce OpenAI's dependence on Nvidia's GPUs. The chip is still being tested, but early results are promising.

Main themes

• AI Hardware
• Custom Chips
• Inference Processors
• Nvidia Competition
• Broadcom Partnership
• AI Accelerators

What commenters say

• The development of custom chips like Jalapeño is crucial for optimizing inference systems and reducing costs.
• Memory bandwidth is a significant bottleneck in current hardware, and increasing it is essential for improving performance.
• Apple's approach to AI hardware is flawed due to their focus on unified SOC memory, which limits their ability to compete with Nvidia.
• Google's decision to partner with MediaTek instead of Broadcom may be driven by cost considerations, but it could compromise performance.
• The use of custom chips like Jalapeño is a strategic move to reduce dependence on Nvidia and improve efficiency in AI workloads.
• The distinction between training and inference costs is becoming more important, with inference costs potentially surpassing training costs as AI adoption increases.
• Broadcom's experience in the ASIC design space and their allocation agreements with TSMC make them a valuable partner for companies like OpenAI and Google.
• The development of custom chips for AI workloads is a complex process that requires careful consideration of factors like memory bandwidth, power consumption, and cost.