Mistral NeMo (mistral.ai)

Article summary

Mistral AI has released Mistral NeMo, a 12B model built in collaboration with NVIDIA, offering a large context window of up to 128k tokens and state-of-the-art reasoning, world knowledge, and coding accuracy in its size category. The model is designed for global, multilingual applications and is trained on function calling, with a large context window, and is particularly strong in multiple languages. Mistral NeMo is easy to use and a drop-in replacement in any system using Mistral 7B. The model is available under the Apache 2.0 license, with pre-trained base and instruction-tuned checkpoints available on HuggingFace.

Main themes

• Mistral NeMo release
• AI model performance
• Multilingual applications
• Model size and complexity
• Quantization and inference

What commenters say

• The model's performance is impressive, but comparing it to smaller models may be unfair due to its larger size.
• Some commenters disagree, arguing that smaller models like Llama 3 8B are still competitive and useful for certain tasks.
• The use of quantization-aware training for FP8 inference is seen as a significant advantage, allowing for efficient deployment on certain hardware.
• There is debate about the optimal model size, with some arguing that 7B models are no longer sufficient for complex tasks, while others see value in smaller models for specific applications.
• The pricing of the hosted version of Mistral NeMo is seen as relatively affordable, potentially due to a subsidy from NVIDIA.
• Some commenters are concerned about 'parameter creep', where models are compared to smaller ones despite having significantly more parameters.
• The choice of model size depends on the specific use case, with factors like inference speed, RAM requirements, and context window size influencing the decision.
• There is a need for standardization in model sizes, with some suggesting tiers like 3B for edge devices, 20-30B for consumer GPUs, and 70B+ for high-end applications.