Ask HN: How are you doing RAG locally?

Article summary

The article asks how people are implementing RAG (Retrieve, Augment, Generate) locally with minimal dependencies for internal code or complex documents. The author is looking for solutions that use vector databases, semantic search, knowledge graphs, or hypergraphs. The discussion revolves around various tools and techniques for implementing RAG locally. Several commenters share their experiences and recommendations for different approaches.

Main themes

• RAG implementation
• Vector databases
• Semantic search
• Code search
• Information retrieval
• Local deployment

What commenters say

• Using a vector database like Qdrant or Faiss can be an effective way to implement RAG locally for certain use cases.
• BM25 and trigram-based search methods are often preferred over embeddings-based search for code search and other applications.
• Some commenters argue that Elasticsearch and Opensearch are complicated to operate and have become bloated, while others find them simple to use.
• Static embedding models can be used to improve the speed of search queries, especially when combined with techniques like quantization and caching.
• Hybrid search approaches that combine multiple methods, such as BM25 and vector search, can be effective in certain scenarios.
• Some argue that embeddings-based search has not yet surpassed traditional methods like BM25 and tf-idf in terms of effectiveness.
• Local deployment of RAG can be achieved using various tools and libraries, including SQLite, Postgres, and specialized libraries like Meilisearch and Manticore.