There is minimal downside to switching to open models (marble.onl)

Article summary

The article discusses the potential downsides of switching to open models, particularly in the context of large language models (LLMs). The author notes that while there may be some performance differences between open and proprietary models, the gap is narrowing. The author also mentions that running open models locally or in the cloud can be a viable option, especially with the availability of good coding harnesses and relatively affordable hardware. The article concludes that the professional penalty for switching to open models may be minimal.

Main themes

• Open models vs proprietary models
• Large language models
• Hardware and infrastructure
• Professional trade-offs
• Data privacy and security

What commenters say

• Investing in local inference may be a bad idea at current prices, but it could be a necessary step for long-term predictability and control.
• The performance difference between open and proprietary models is not significant enough to justify the costs and risks associated with proprietary models.
• Running open models locally or in the cloud can be a viable option, especially with the availability of good coding harnesses and relatively affordable hardware.
• The constant release of new proprietary models creates a honeymoon effect, where users feel productive and impressed, but this effect wears off as issues and limitations become apparent.
• There is a risk that proprietary model providers may intentionally degrade their models over time, making local inference a more attractive option.
• The potential for export controls or other restrictions on access to proprietary models makes investing in local inference a prudent decision.
• The benefits of using open models, such as increased control and autonomy, outweigh the potential drawbacks, including the need to invest in hardware and infrastructure.
• The idea that one must use the latest and greatest proprietary models to be productive is a marketing myth, and open models can be just as effective for many use cases.