Show HN: Echo – Fable-level results at 1/3 the cost using open-weight models

Article summary

The author has built an AI system called Echo, which combines the outputs of multiple open-weight models to achieve better results than any individual model. Echo decides how much computation to allocate, which models to use, and how to combine their outputs for each request. The system has been evaluated on a set of benchmarks and has achieved results comparable to Fable, a stronger comparison system, at around one third of the inference cost. The author has made a chat interface and API available for testing.

Main themes

• AI model combination
• Open-weight models
• Inference cost reduction
• Benchmark evaluation
• Model routing
• Ensemble methods

What commenters say

• The lack of transparency in Echo's routing decision-making process may be a significant drawback for potential customers who require observability and control over the models used.
• Combining multiple models can lead to better performance, but it is not a new concept and has been explored in ensemble methods and other areas of machine learning.
• Some commenters are skeptical about the claim of achieving Fable-level results at a lower cost, citing the need for more rigorous benchmarking and evaluation.
• The use of open-weight models and model routing can be an effective way to reduce inference costs and improve performance, but it also introduces new challenges and complexities.
• The comparison to other systems, such as OpenRouter and Sakana Fugu, highlights the need for more clarity on how Echo differs from existing solutions.
• The requirement for a signup and potential lack of transparency in the privacy policy may deter some users from trying the system.
• Some commenters believe that the concept of combining models is not particularly interesting or novel, and that the focus should be on more practical and applied problems.
• The development of systems like Echo and others in the space may be driven by the need to reduce costs and improve performance, but it also raises questions about the potential for abuse and the importance of transparency and accountability.