Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

Article summary

The article asks if anyone has replaced cloud-based coding tools like Claude/GPT with local models, and if so, to share their setup and performance. Commenters discuss their experiences with local models, including their performance, power consumption, and cost. Some have successfully used local models for coding, while others have found them to be slow or inconsistent. The discussion also touches on the potential for distributed AI models and the challenges of running them on local hardware.

Main themes

• Local AI models
• Cloud vs local coding tools
• Performance and power consumption
• Distributed AI models
• Coding productivity

What commenters say

• Some users have successfully replaced cloud-based coding tools with local models, achieving comparable performance and productivity.
• Local models can be slow and inconsistent, making them less reliable than cloud-based alternatives for certain tasks.
• The high upfront cost of hardware and electricity consumption are significant barriers to adopting local AI models for coding.
• Distributed AI models could potentially offer a solution for users who want to contribute their computing resources to train and run AI models, but significant technical challenges need to be overcome.
• Running local AI models can be cost-effective, especially for users with access to cheap electricity or solar power.
• The performance of local AI models can be improved with optimized hardware and software configurations, such as using specific GPUs or quantization techniques.
• Some users prefer to use local models for coding due to concerns about data privacy and security, while others prioritize the convenience and scalability of cloud-based tools.
• The development of more efficient and powerful local hardware, such as the Strix Halo, is making it possible to run larger and more complex AI models locally.