Ask HN: Is anyone doing anything cool with tiny language models?

Article summary

The article asks if anyone is doing anything cool with tiny language models, specifically in the 0.5B-3B range, and available on Ollama. The author is looking for examples of tooling that uses these models as part of a workflow. The discussion revolves around the use of small language models for various tasks, such as code completion and anonymization. Some commenters share their experiences with training and using small models locally.

Main themes

• Tiny language models
• Code completion
• Anonymization
• Local model training
• Model size and performance
• Practical applications

What commenters say

• Small language models can be trained and used locally for tasks like code completion and anonymization, with some models being small enough to be hosted as a static web app.
• The size of a model is not a significant barrier to training at home, with some commenters suggesting that models in the 0.5B-3B range can be trained on modern hardware or affordable cloud services.
• Some commenters believe that small language models can be effective for specific tasks, such as generating bash one-liners or creating human-readable regex patterns, while others are skeptical about their accuracy and reliability.
• The use of small language models can have benefits, such as reducing the amount of data that needs to be sent to larger models, and allowing for more focused attention on specific tasks.
• There are existing tools and plugins that utilize small language models for tasks like code anonymization and completion, and some commenters are interested in exploring these tools further.
• The performance of small language models can be sufficient for certain tasks, with some commenters reporting acceptable speeds and accuracy, while others note that larger models may still be necessary for more complex tasks.