So you want to use OpenRouter? (mmoustafa.com)

Article summary

The article discusses the challenges of using OpenRouter, a service that routes requests to various providers of open-source models. The author shares their experience of running an AI assistant on OpenRouter and highlights several pitfalls, including inconsistent performance across providers, varying levels of quality, and issues with tool calling and parsing. The author provides a list of things they wish they had known before using OpenRouter. The article aims to inform users about the potential issues with using OpenRouter and similar services.

Main themes

• OpenRouter challenges
• LLM provider quality
• Vendor neutrality
• API security
• Model testing and filtering
• Service reliability and consistency

What commenters say

• Using a single vendor can avoid the complexity and issues associated with OpenRouter and similar services.
• Vendor neutrality is not worth the potential drawbacks of using a meta-provider like OpenRouter.
• The problems with OpenRouter are not unique to the service, but rather a result of the underlying issues with serving LLMs in general.
• First-party providers are often more reliable and consistent than third-party providers.
• OpenRouter's abstraction layer cannot fix the underlying issues with provider quality and consistency.
• The service's billing limits and API key protections are inadequate, leaving users vulnerable to unexpected charges and security risks.
• Testing and filtering providers is crucial to achieving reliable results with OpenRouter.
• The benefits of using OpenRouter, such as easy testing and comparison of models, may outweigh the drawbacks for some users.