Ollama is now powered by MLX on Apple Silicon in preview (ollama.com)

Article summary

Ollama is now powered by MLX on Apple Silicon in preview, allowing for faster performance on macOS devices. This update unlocks new performance to accelerate demanding work, including personal assistants and coding agents. The preview release of Ollama accelerates the Qwen3.5-35B-A3B model, with sampling parameters tuned for coding tasks. Ollama on Apple silicon is built on top of Apple's machine learning framework, MLX, to take advantage of its unified memory architecture.

Main themes

• Local LLMs
• Apple Silicon
• MLX Framework
• Coding Assistants
• Performance Optimization
• Cloud vs Local LLMs

What commenters say

• Local LLMs are the future, offering more security and potentially lower electricity usage, but may not replace cloud LLMs entirely.
• Some users prefer local LLMs for coding tasks to avoid company tracking and management control, and are willing to pay for good hardware.
• Cloud LLMs will always be faster and smarter than local LLMs, making them preferable for many use cases, especially those requiring high intelligence and throughput.
• The development of open models and specialized models may allow local LLMs to compete with cloud LLMs in the future, potentially changing the landscape of the industry.
• Most users do not need frontier model performance, and smaller, faster models may be sufficient for their needs, such as grammar and spelling correction or simple fact lookup.
• The free version of ChatGPT is often insufficient and prone to hallucinations, making it less reliable than paid models or local LLMs with good harness and toolsets.
• Local LLMs may not be suitable as a direct replacement for cloud LLMs, but can be effective when combined with a robust toolset and used in conjunction with external data sources.
• The trade-off between local and cloud LLMs depends on various factors, including the specific use case, required performance, and user preferences.