Thinking Machines Lab, the San Francisco-based AI startup started and led by former OpenAI chief technology officer Mira Murati, is challenging the dominant paradigm of cloud-based AI frontier models with the release yesterday of Inkling 2 Mini, an open-weights multimodal model engineered to run offline on a 32GB MacBook Pro or a single desktop GPU.
At the same time, Inkling 2 Mini rivals much larger systems on software engineering and tool-use benchmarks.
At 26.4 billion total parameters and a ~20GB memory footprint, Inkling 2 Mini represents a deliberate shift in how AI labs extract reasoning performance from compact hardware. Rather than scaling parameter counts upward, Thinking Machines relies on a “looped transformer” architecture that cycles data through its middle layers multiple times per token.
The result is a U.S.-built, Apache 2.0-licensed model that ties Alibaba’s dense 27B flagship on aggregate intelligence benchmarks while running roughly four times faster and six times cheaper per task—a combination that has already drawn rapid support from local-AI runtimes and enthusiastic early reviews across the developer community.
Technology: How looping buys deeper reasoning without bigger weights
Standard transformer models process each token in a single, linear pass from the first layer to the last. If a problem requires more sequential reasoning steps than the model’s depth allows, the model must externalize its thinking by generating long chains of thought (CoT) in text. That approach burns through output tokens, increases latency, and inflates inference costs.
Inkling 2 Mini takes a different path. According to the Inkling 2 Mini model card, the system is a 24-layer, decoder-only sparse Mixture-of-Experts (MoE) model that routes each token to 8 of 128 experts, activating 3.4 billion parameters on a standard pass. The architectural twist lies in how those 24 layers execute: the first six layers (the prelude) and the final six layers run once, while the middle 12 layers run three times using shared weights. Think of it like a human engineer pausing to mentally double-check a complex equation three times before writing down the next line of code, rather than scribbling out three pages of scratch work.
Those three passes yield 48 effective layer applications per token—and 5.4 billion effective active parameters—without increasing the model’s storage footprint. To keep information flowing cleanly across iterations, Thinking Machines implemented attention residuals, allowing each pass to read a gated “message” from the previous pass’s middle layers alongside the initial prelude output. In internal scaling tests, the company found that a non-looping baseline required 140% more inference compute to match the looped model’s loss on 512K-token agentic traces.
To make the model practical on consumer silicon, Thinking Machines pruned two-thirds of the experts mid-training (from 384 down to 128) and “healed” the network by distilling 1 trillion tokens from a larger internal Inkling model, followed by supervised fine-tuning and 2.2 trillion tokens of reinforcement learning. Crucially, the team trained and deployed the first 20 layers natively in 4-bit NVFP4 precision and the final four layers in 8-bit MXFP8 precision. Because the weights ship in the exact numeric format used during training, developers avoid the accuracy degradation that typically plagues post-training quantization.
Looping does raise an important safety question: if a model performs more reasoning internally without writing it down, does it become harder to oversee? Alongside Inkling 2 Mini, Thinking Machines released a companion variable-loop checkpoint, TML-d24-varloop-research, trained for one to five passes, and partnered with the National Deep Inference Fabric (NDIF) and Goodfire to study mechanistic interpretability. In Thinking Machines’ preliminary monitorability tests, looping extended the model’s no-CoT reasoning horizon from 16 seconds at one pass to 37 seconds at three passes and cut CoT length by more than half—yet the remaining reasoning traces stayed human-legible, and the model showed no upward trend in covert sabotage on the LinuxArenacontrol benchmark.
Product: Benchmarks, local performance, and how it compares to peers
Inkling 2 Mini is natively multimodal, accepting UTF-8 text and pixel-based images (optimally between 40px and 4096px) across a 1-million-token context window and outputting text. Thinking Machines demonstrated its offline capabilities by running the model in airplane mode on a 48GB M5 Pro MacBook Pro to build a playable flight simulator in minutes, as well as powering 20 persistent autonomous villagers simultaneously in a local town simulation.
On standardized evaluations, Inkling 2 Mini punches well above its weight class in agentic coding and tool use. It leads all sub-100B open-weights models on DeepSWE v1.1 (48.2%), SWE-bench Pro v2 Hard (77.9%), and Vals Index v2.2 (46.5%), while scoring 79.2% on SWE-bench Verified and 85.3% on DeepSearchQA. On Stanford’s intelligence-per-joule benchmark running on an NVIDIA DGX Spark, Inkling 2 Mini scored 0.703, comfortably ahead of Qwen3.6-35B-A3B (0.430) and Gemma-4-26B-A4B (0.159).
However, looping cannot conjure facts out of thin air. Because factual recall scales primarily with total parameter count, Inkling 2 Mini scores -8.6 on AA-Omniscience v1.1 (with a 63.4% non-hallucination rate), trailing denser peers like Qwen3.8-27B (-6.4 and 71.4%). Thinking Machines also notes in its documentation that the model trails Qwen3.8-27B on multi-turn chat (50.0% on MultiChallenge) and multi-constraint instructions (33.4% on ComplexConstraints v1.1), and can occasionally fall into repetitive debugging loops or produce clipped phrasing in non-English languages such as Chinese.
To evaluate where Inkling 2 Mini sits in the current open-weights landscape, the table below compares it against three prominent peers in the 26B–31B class: Alibaba’s Qwen3.8-27B, Google DeepMind’s Gemma 4 31B, and Meta’s Muse Glimmer.
|
Model & Developer |
Architecture (Active / Total) |
AA Intelligence Index (v5.0) |
DeepSWE v1.1 / SWE-bench Verified |
Hosted API Pricing (Input / Output per 1M) |
License |
Local / Offline Support |
|---|---|---|---|---|---|---|
|
Inkling 2 Mini (Thinking Machines) |
Looped MoE (5.4B eff. / 26.4B) |
20.1 |
48.2% / 79.2% |
$0.15 / $0.50 (first-party); from $0.08 / $0.35 on OpenRouter |
Apache 2.0 (with AUP) |
Native ~20GB NVFP4/MXFP8; 32GB+ Macs (Ollama, EXO), DGX Spark, 16GB+ via GGUF |
|
Qwen3.8-27B (Alibaba) |
Dense (27.0B / 27.0B) |
20.1 |
42.2% / 77.8% |
$0.50 / $3.00 (Alibaba Cloud); from $0.08 / $0.50 via third parties |
Apache 2.0 |
Local/offline capable via llama.cpp/vLLM, but dense 27B requires heavier compute per token |
|
Gemma 4 31B (Google DeepMind) |
Dense (30.7B / 30.7B) |
4.4 |
0.3% / 46.1% |
$0.20 / $0.40 (AI Studio); from $0.10 / $0.34 on OpenRouter |
Apache 2.0 |
Local/offline capable via standard GGUF/MLX quantizations |
|
Muse Glimmer (Meta) |
Dense (30.0B / 30.0B) |
6.3 |
4.4% / 76.0% |
$0.30 / $1.20 on DeepInfra ($0.32 / $1.35 median) |
Apache 2.0 |
Local/offline capable via standard quantizations; 131K context |
The sharpest contrast emerges against Qwen3.8-27B. While both models tie at 20.1 on the Artificial Analysis Intelligence Index, Artificial Analysis reports that Inkling 2 Mini uses roughly 2.8 times less compute, runs four times faster (9.0 minutes per task versus 38.2 minutes), and costs six times less per task ($0.59 versus $3.53 at first-party API rates). Against Gemma 4 31B and Muse Glimmer, Inkling 2 Mini holds a commanding lead on agentic software engineering and long-context retrieval (1M tokens versus 262K for Gemma 4 31B and 131K for Muse Glimmer), making it a far stronger candidate for autonomous coding workflows.
Licensing: Apache 2.0 keeps enterprise adoption straightforward
Thinking Machines released Inkling 2 Mini’s full weights on Hugging Face under the Apache 2.0 license, paired with a standard Model Acceptable Use Policy.
In plain terms, Apache 2.0 is one of the most permissive open-source licenses in software. It allows startups, researchers, and Fortune 500 enterprises to download, modify, fine-tune, redistribute, and embed the model inside proprietary commercial products royalty-free, provided they include the original copyright notice and license text. Unlike copyleft licenses such as the GPL, Apache 2.0 does not force companies to open-source their own downstream applications.
That puts Inkling 2 Mini on equal legal footing with Qwen3.8-27B, Gemma 4 31B, and Meta’s Muse Glimmer—all of which also use Apache 2.0—while avoiding the commercial friction of custom “open-weight” agreements seen on larger frontier models, such as Mistral’s Ecosystem Open-Weight license or Meta’s Muse Model License for its 816B Muse Spark Open, which impose separate commercial terms or revenue thresholds on model-as-a-service providers. For enterprise legal and procurement teams, Apache 2.0 removes a major hurdle to private, on-premises deployment.
Community reactions: Local-AI builders welcome a U.S. contender
Within 24 hours of launch, Inkling 2 Mini generated substantial traction on X, driven by rapid day-one integrations from local-inference tooling providers and hands-on tests from independent developers.
Thinking Machines kicked off the conversation from its official account, @thinkymachines, writing: “Some of you thought that Inkling 2 Small was too big at 594B parameters. We heard you: Inkling 2 Mini is a looped transformer with just 26B parameters, small enough to run on your laptop, powerful enough for agentic work with a score of 20.1 on AA Index.”
Infrastructure partners quickly followed with local speed benchmarks. @UnslothAI, which published quantized GGUF builds alongside llama.cpp support, highlighted its accessibility: “Run Inkling 2 Mini, the strongest 26.4B model that runs on 16GB RAM. Thinking Machines’ Apache 2.0 MoE model uses a looped transformer arch & scores 48.2% on DeepSWE v1.1, the best for size.”
Apple silicon specialists @exolabs shared early throughput numbers from their collaboration with the lab: “We worked with @thinkymachines to make Inkling 2 Mini run fast locally: – 210 tok/sec decode, 3620 tok/sec prefill on 96GB M5 Ultra Mac Studio – 125 tok/sec decode… on 48GB M5 Max… It built a native weather app 1.8x faster than Claude Sonnet 5.5… and 11x cheaper… The looping architecture is ideal for local devices…”
Independent testers also praised its agency on single-GPU setups. After using the model to autonomously generate a motion graphic about its own specs, developer @KyleHessling1 wrote: “Inkling 2 mini just made this motion graphic about itself… Motion design is something that really wrecks tiny models; Inkling just crushed this flow autonomously… 104tps on the 5090… this level of intelligence in a small MOE is astounding… the most impressive small model I’ve tested from anyone outside of Qwen… Beholding a USA contender for the single GPU frontier has me shedding tears of red white and blue.”
A minority of early testers tempered the enthusiasm, noting occasional task failures in game-mechanics generation, early quantization quirks across nascent looped-model runtimes, and the fact that cloud frontier models still hold a clear advantage on the most demanding knowledge and multi-turn workloads.
Background: Thinking Machines’ rapid march from stealth to local AI
Inkling 2 Mini marks the latest milestone in an unusually fast product cadence for Thinking Machines Lab. Founded in early 2025 by former OpenAI CTO Mira Murati alongside OpenAI co-founder John Schulman and other veteran researchers, the startup raised a historic $2 billion seed round at a $12 billion valuation in July 2025 with a mandate to build customizable, open-science AI systems.
The company debuted its first commercial product in October 2025 with Tinker, a Python-native fine-tuning API designed to let researchers control training loops and reinforcement learning without managing distributed GPU clusters. In May 2026, it previewed TML-Interaction-Small, a full-duplex multimodal system capable of simultaneous voice and video interaction with 0.40-second turn-taking latency.
Thinking Machines then turned its attention to open-weights foundation models. In July 2026, it released the original 975B-parameter Inkling under Apache 2.0, followed two weeks later by the 276B-parameter Inkling-Small, which matched its larger sibling’s performance at roughly a quarter of the size. Earlier this month, on November 5, the company opened its second generation with Inkling 2 Small, a 594B-parameter (17B active) enterprise powerhouse that scored 31 on the Artificial Analysis Intelligence Index.
With Inkling 2 Mini, Thinking Machines has finally delivered a model that lives up to the promise of local deployment—not by stripping away agentic capability, but by rethinking how a compact transformer spends its compute. For developers and enterprises looking to run private, high-agency coding assistants on laptops or single workstations, looped transformers have officially moved from academic curiosity to practical reality.
Credit: Source link


























