Ollama now supports AMD graphics cards (ollama.com)

Article summary

Ollama now supports AMD graphics cards in preview on Windows and Linux, allowing all features of Ollama to be accelerated by AMD graphics cards. This support is currently available for download on Linux and Windows. The article does not provide detailed performance comparisons or implementation details. Support for more AMD graphics cards is planned for the future.

Main themes

• AMD graphics card support
• Ollama development
• GPU acceleration
• CUDA and ROCm comparison
• OpenCL and Vulkan discussion

What commenters say

• The support for AMD graphics cards in Ollama reduces NVIDIA's competitive advantage and the moat of CUDA.
• The performance of AMD graphics cards is not yet sufficient to surpass NVIDIA's high-end cards, but may be competitive in the inference market.
• The use of OpenCL and ROCm allows for more cross-platform compatibility, but may not be as performant as CUDA or Vulkan.
• The development of Ollama is seen as a positive step towards democratizing access to AI models, but some commenters are puzzled by the perception of Ollama as a large, unique entity.
• Some commenters believe that AMD's open-source strategy will pay off and allow them to catch up with NVIDIA in the hardware market.
• Others argue that the main barrier to adopting AMD graphics cards is not hardware, but rather software support and compatibility issues.
• The discussion around OpenCL and Vulkan highlights the fragmentation in the GPU compute market and the need for cross-platform compatibility.
• Some commenters note that HIP allows for CUDA code to run on AMD GPUs with minimal modifications, but this is not a widely known or reliable solution.