news.volyx.in

Hello OLMo: A truly open LLM (blog.allenai.org)

398 points by tosh · 859 days ago · 67 comments on HN

Article summary

The Allen Institute for AI has released OLMo, a truly open large language model, along with its pre-training data and training code. This release aims to provide the industry with an opportunity to understand how AI models are created and to advance the science of language models collectively. OLMo is designed to aid researchers in training and experimenting with large language models, and it is available for direct download on Hugging Face and GitHub. The model is built on Ai2's Dolma set, which features a three trillion token open corpus for language model pretraining.

Main themes

  • Open Language Models
  • AI Transparency
  • Model Training
  • Research Collaboration
  • AI Ethics
  • Licensing and Copyright

What commenters say

  • The release of OLMo is significant because it provides a truly open large language model that can be audited and understood, unlike closed models from other companies.
  • The lack of transparency in closed models can lead to biased or manipulated information, which is a concern for consumers of information.
  • Some commenters question the effectiveness of OLMo compared to other models, suggesting that it may not be as good as expected given its size and compute budget.
  • The use of proprietary licenses for open models can be problematic, as it may restrict the use of the model and create uncertainty around copyright and fair use.
  • The inclusion of unlicensed, copyrighted data in some open models raises ethical and legal concerns, highlighting the need for more careful curation and licensing of training data.
  • The comparison of OLMo to other models, such as Mistral 7B, is notable for its absence, leading some to wonder about the motivations behind the omission.
  • The requirement to report the intended use of OLMo derivatives may be seen as a restriction on the use of the model, and some commenters are unsure about the implications of this requirement.
  • The release of OLMo is seen as a positive step towards more open and collaborative research in AI, but some commenters are skeptical about the potential impact and effectiveness of the model.