news.volyx.in

Running Gemma 4 locally with LM Studio's new headless CLI and Claude Code (ai.georgeliu.com)

407 points by vbtechguy · 147 days ago · 103 comments on HN

Article summary

The article describes how to run Google's Gemma 4 model locally on a Mac using LM Studio's new headless CLI and Claude Code. The model's mixture-of-experts architecture allows it to run efficiently on hardware that could not handle a dense 26B model. The article provides step-by-step instructions on how to install and configure LM Studio, download the Gemma 4 model, and run it locally. This setup allows for local inference, which can be beneficial for tasks like code review and testing prompts.

Main themes

  • Local AI model inference
  • Gemma 4 model
  • LM Studio
  • Claude Code
  • Mixture-of-experts architecture
  • AI model optimization

What commenters say

  • Using Claude Code with local models can be a viable alternative to cloud-based models, but it may have limitations and require additional configuration.
  • Some users have reported issues with running Gemma 4 with Claude Code, but using Ollama or other tools can resolve these issues.
  • The choice of coding agent harness, such as Claude Code or OpenCode, depends on personal preference and specific use cases, with some users finding Claude Code to be more capable despite its limitations.
  • Running local models can be more cost-effective and provide better performance for certain tasks, but may require significant computational resources and expertise to set up and optimize.
  • The mixture-of-experts architecture in models like Gemma 4 can improve performance and efficiency, but may not necessarily reduce memory usage.
  • Some users prefer open-source tools like Pi and OpenCode over Claude Code, citing their flexibility and customizability.
  • The use of local models with Claude Code may be affected by the upcoming updates from Anthropic, which could potentially limit the use of non-Anthropic models with the tool.
  • The quality of local models, including Gemma 4, is still inferior to that of cloud-based models like Opus, which may limit their adoption in certain industries and applications.