A guidance language for controlling LLMs (github.com)

Article summary

A new programming paradigm called Guidance has been introduced to control large language models. Guidance allows users to constrain the output of language models using regular expressions, context-free grammars, and other methods. This enables more efficient and high-quality output from language models, reducing latency and cost. Guidance also provides a Pythonic interface for working with language models and supports various backends.

Main themes

• Guidance programming paradigm
• Large language models
• Constrained output
• Efficient generation
• Reliability and production use
• Commercial endpoints and APIs
• Hype and satire
• Technological enthusiasm and skepticism
• Model quality and control tradeoffs
• Language model applications and limitations

What commenters say

• Guidance is seen as a competitor to LangChain, but is more low-level and less opinionated.
• Some commenters are skeptical about the effectiveness of Guidance with remote LLMs like OpenAI.
• Others believe that Guidance can be useful even with commercial endpoints like OpenAI, despite requiring multiple calls.
• There are concerns about the reliability of LLMs for production use cases, with some experiencing high failure rates.
• The use of Guidance and similar technologies is seen as a way to improve the control and acceleration of LLM output.
• Some commenters are amused by the enthusiasm and terminology used in the discussion, seeing it as satire or indicative of a hype phase.
• Others defend the excitement and interest in the technology, arguing that it is novel and compelling.
• There is a perceived tradeoff between model quality and the ability to control and accelerate the output, with some prioritizing one over the other.