Bonsai 27B: A 27B-Class model that runs on a phone (prismml.com)

Article summary

PrismML has announced Bonsai 27B, a 27B-class model that can run on a phone, with two variants: Ternary Bonsai 27B and 1-bit Bonsai 27B. The model achieves a high level of intelligence density, allowing it to fit on devices with limited memory. Bonsai 27B is capable of multi-step reasoning, structured tool calls, vision tasks, and computer-use agentic loops. The model is available under the Apache 2.0 License and can be run natively on Apple devices and NVIDIA GPUs.

Main themes

• AI model compression
• Low-bit models
• Ternary and binary weights
• Intelligence density
• Edge AI
• Model deployment

What commenters say

• The concept of 1-bit models is misleading, as they often use ternary weights with three values: -1, 0, and 1, which can be more accurately represented as 1.58 bits per weight.
• The use of ternary and binary weights allows for significant model compression, making it possible to deploy large models on devices with limited memory.
• Some commenters have expressed skepticism about the effectiveness of the model, citing potential issues with reasoning loops and performance compared to other models.
• The model's ability to run on a phone is seen as a significant achievement, enabling new use cases for edge AI and local model deployment.
• There is disagreement about the best way to quantify the model's performance, with some arguing that the 1-bit variant is less effective than the ternary variant.
• The model's potential applications are vast, including persistent on-device agents, assistants that work offline, and assistants that reason over private local data.
• Some commenters have reported difficulties in getting the model to work with certain frameworks and devices, highlighting the need for further support and development.