Ask HN: How do I train a custom LLM/ChatGPT on my own documents in Dec 2023?

Article summary

The article asks about the best approach to train a custom large language model (LLM) or ChatGPT on a set of documents in December 2023. A previous thread on the topic is mentioned, but it may be outdated. The question is about how to teach an LLM to answer questions using a custom set of documents. This can be achieved through methods like Retrieval-Augmented Generation (RAG) rather than training a model from scratch.

Main themes

• Custom LLM Training
• RAG vs Training
• Cloud Services
• Cost Effectiveness
• LLM Limitations
• Alternative Frameworks

What commenters say

• Using AWS Bedrock is a viable option for creating a knowledge base with custom documents, but it can be expensive.
• RAG is a suitable approach for utilizing a custom set of documents with an LLM, rather than training a model from scratch.
• Some commenters find AWS Bedrock and other cloud services to be prohibitively expensive for small projects or hobbyists.
• LlamaIndex is preferred over Langchain for RAG due to its simplicity and performance.
• Fine-tuning a pre-trained model like GPT-3.5 can be a more cost-effective option than using a custom model.
• The effectiveness of a custom knowledge base is limited by the LLM's context window and the quality of the document embeddings.
• Alternative frameworks like Cheshire Cat, Haystack, and Langroid offer different approaches to building AI assistants with custom documents.