PrismML has released Ternary Bonsai 2 27B, a ternary-weight version of Qwen3.8 27B. The language model occupies 5.93 GB, against 53.80 GB in FP16. PrismML reports that it keeps 98.2% of the parent model’s average across 20 benchmarks. The model accepts text and images and supports a 262K-token context. PrismML demos it driving Cline coding agents and computer use on an RTX 5090. It arrives 2 months after the first Bonsai 27B, whose ternary variant retained about 95%.
Is it deployable? Yes. The Apache 2.0 weights run today on a 16 GB laptop or a single 24 GB GPU. You need PrismML’s llama.cpp fork or its MLX runtime.
What is Ternary Bonsai 2 27B?
The model keeps the Qwen3.8 27B architecture unchanged. It has 27.36B parameters. That splits into a 24.35B language backbone, 2.54B in embeddings and LM head, and a 0.47B vision tower. The backbone uses hybrid attention, with about 75% linear-attention and 25% full-attention layers.
Ternary weights cover embeddings, attention projections, MLP projections and the LM head. Only 26.2M parameters, or 0.0976%, stay in higher precision. Those are the recurrent state path and normalization weights. In GGUF, the vision tower ships separately as a 0.63 GB file, loaded only for image input.
How Does the Ternary Format Work?
Each weight takes 1 of 3 values: -1, 0 or +1. Every group of 128 weights shares 1 FP16 scale. A ternary value carries log2(3), or about 1.585 bits. Adding 16 scale bits per 128 weights gives 1.71 bits per weight. Counting the high-precision tensors brings the model to 1.72.
Real kernels need a packed layout, so the whitepaper describes 2 GGUF packings. PTQ1_0 packs trits densely at 1.76 bits per weight and 5.93 GB. PQ2_0 stores each trit in a 2-bit slot at 7.25 GB, which is cheaper to unpack.
Weights are also stored in a rotated basis. PrismML applies a blockwise Hadamard rotation with block size 1,024 before ternary assignment. The runtime applies the matching transform to activations before each multiply. The whitepaper cites SpinQuant for this idea. PrismML does not publish how it assigns the ternary values.
Credit: Source link
