news.volyx.in

OpenAI compatibility (ollama.ai)

643 points by Casteil · 922 days ago · 188 comments on HN

Article summary

Ollama now supports the OpenAI Chat Completions API, allowing users to run local models with OpenAI-compatible tooling and applications. This experimental support enables the use of OpenAI-compatible libraries and frameworks, such as the OpenAI Python and JavaScript libraries, with Ollama. The article provides examples of using Ollama with various libraries and frameworks, including Vercel AI SDK and Autogen. Future improvements are planned, including support for embeddings, function calling, and vision.

Main themes

  • OpenAI API compatibility
  • Ollama
  • LangChain
  • abstraction layers
  • lock-in
  • technical debt
  • API standards
  • interoperability
  • community standards
  • marketing and technical problems

What commenters say

  • Some commenters question the need for third-party projects to conform to OpenAI's API contract, citing the existence of abstraction layers like LangChain.
  • Others argue that using an abstraction layer can create lock-in and technical debt, and that a thin layer wrapping the underlying LLM is a better approach.
  • There are concerns about Ollama's installer and its use of dark patterns, such as silently adding login items and spawning persistent processes.
  • Some users have had negative experiences with LangChain, finding its abstractions to be overly verbose and difficult to debug and maintain.
  • Others have had positive experiences with LangChain, citing its ease of use and the benefits of its integration with other tools.
  • There is a debate about the potential risks and benefits of adopting OpenAI's API as a community standard, with some arguing that it could lead to lock-in and others seeing it as a way to reduce friction and improve interoperability.
  • Some commenters suggest that a robust specification and test suite are needed to ensure that implementations conform to the OpenAI API standard.
  • The use of OpenAI's API is seen as a marketing and technical problem, with some arguing that it can limit the ability to add new features and create a cognitive burden on developers.