🔍 Read the full analysis: How To Bring Inkling 2 Mini Into Your Local AI Setup on ThorstenMeyerAI.com
Get the latest gadgets delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
TL;DR
Thinking Machines Lab has released Inkling 2 Mini, a multimodal mixture-of-experts model the company describes as having about 26 billion total parameters, with about 3 billion active per token. It supports several inference runtimes and includes a research checkpoint with adjustable layer loops, but the supplied material does not establish independent benchmark results or practical hardware requirements.
Thinking Machines Lab has released Inkling 2 Mini, a multimodal mixture-of-experts model intended for use with local AI tools. The company describes it as having about 26 billion total parameters, with about 3 billion active per token, and says it works with Transformers, llama.cpp and MLX; SGLang support is listed for datacenter inference. The release also includes a research checkpoint with adjustable layer-loop passes, as detailed in the original analysis, while independent performance checks and concrete hardware requirements remain unavailable in the supplied material.
Inkling 2 Mini accepts text and image input and is described by the company as having a 1 million-token context window. Its architecture contains 24 physical decoder layers. The first and last six layers run once, while the 12 middle layers run three times, producing 48 logical layer visits per token. The company says this reuses the middle layers’ weights rather than storing a separate set for every logical visit.
The model is a mixture of experts: at each layer visit, a routing system selects eight experts from a pool of 128. Thinking Machines says the expert weights are distributed in NVFP4 and MXFP8 formats and that a DSpark drafter is included to help speed generation with a small memory cost. These are design and vendor statements; the supplied information does not provide independent speed or memory measurements.
The fixed-pass model is listed for Transformers, llama.cpp and MLX, with SGLang support for datacenter deployments. A separate research checkpoint, TML-d24-varloop-research, can change the number of passes. The company says it can run with one or two passes instead of three, potentially increasing speed at a possible cost to output quality.
Inkling 2 Mini for Local AI
Thinking Machines Lab’s new multimodal model combines reused layers with expert routing. Its practical speed, hardware needs and benchmark results still need outside validation.
A loop reuses model layers
The design reuses middle-layer weights to build capacity, while routing selects part of the expert pool at each layer visit.
The first six and last six physical decoder layers run once.
The same 12 middle layers run in three passes, for 48 logical layer visits per token.
At each visit, the model selects eight experts from a pool of 128.
Options for inference
The release offers several runtimes and a research checkpoint with adjustable loop passes.
Multiple runtimes
Transformers, llama.cpp and MLX support the fixed-pass model; SGLang supports datacenter inference.
Fewer passes
The research checkpoint allows one or two passes. The company says this can be faster, with a possible quality trade-off.
Claims need outside checks
The company reports strong benchmark comparisons, but the supplied material does not include independent replication or a full score table.
- Leads in 32 of 48 benchmarks against named models.
- Reported tie with Qwen3.8-27B on Terminal-Bench 4.0.
- Trails on AutomationBench-AA v1.0.6.
- Complete memory requirements and device-specific speed.
- Independent image and long-context performance.
- Quality at reduced loop counts.
What local users should assess
Runtime availability is a starting point; actual usefulness depends on hardware and workload.
What Local Model Users Gain
The release gives people assembling local AI setups another multimodal model to test across several established inference tools. Runtime availability can lower the barrier to trying a model, but it does not guarantee that a particular computer has enough memory or that generation will be responsive. Users will still need to check compatibility, model-weight requirements and performance on their own hardware.
Inkling 2 Mini’s architecture offers a particular trade-off: it reuses weights across repeated layer passes and activates only a subset of experts at each visit. That may allow substantial model capacity without assigning separate weights to every logical layer. However, the technical description says the fixed model still executes all 48 visits, and each visit retains its own key-value and short-convolution state. Weight reuse alone does not establish low total memory use or fast inference.
The claimed million-token context and image input could matter to users working with lengthy material or visual information. Their practical value depends on how accurately the model handles those inputs, how much memory long contexts require, and whether results remain useful as input length grows. The source material provides no independent tests resolving those questions.
Products Worth Considering

Bose QuietComfort Wireless Noise Cancelling Headphones, Sandstone

Apple MacBook Air 13-inch with M5 Chip, 16GB Memory, 512GB SSD, Midnight

LG UltraGear 27-inch QHD OLED Gaming Monitor, 240Hz, 0.03ms, G-SYNC Compatible, FreeSync Premium Pro, HDR True Black 400
As an affiliate, we earn on qualifying purchases.
How Its Layer Loop Works
The central design choice is the reuse of the 12 middle physical layers. Those layers run in three passes, with learned messages carrying information between passes; the company says a learned gate controls how much information is passed forward. This creates 48 logical layer visits across the model while retaining 24 physical decoder layers. The release describes the fixed model as using three passes and the research checkpoint as the version for changing that setting.
Thinking Machines compares Mini with its Inkling 2 Small model. Both are described as executing 48 layer visits per token, but Mini has a 2,304 hidden width, compared with 4,608 for Small, and routes among 128 experts per layer rather than 384. The supplied description says Mini is substantially smaller but gives no complete file size or minimum hardware configuration. It also notes that earlier architectures, including Nanbeige, have explored looping; the release does not provide a detailed comparative evaluation of those approaches.
The company reports competitive benchmark results, including a lead in 32 of 48 benchmarks against Gemma 4 31B, Muse Glimmer, Qwen3.8-27B and Nemotron 3.5 Super VL on the comparisons it describes. It also reports a lead on FrontierCode v2.0, a tie with Qwen3.8-27B on Terminal-Bench 4.0, and a trailing result on AutomationBench-AA v1.0.6. These remain company-reported comparisons: the material provided does not include the full score table or independent replication.
“Inkling 2 Mini is a mixture-of-experts model with 24 decoder layers, but the 12 middle layers run three times in a loop.”
— Thinking Machines Lab, in the supplied release material
Hardware and Results Still Unverified
The supplied material does not specify a minimum hardware configuration, complete model file size, or measured generation speeds across consumer devices. It also does not provide independent testing of image performance, long-context behavior, or output quality. Users therefore cannot determine from these details alone whether the model will fit or run well on a particular local system.
Benchmark claims need further documentation and outside checks. The material gives selected comparisons but not the full scores, test setup or independent replication. It also does not fully assess the quality trade-off when the research checkpoint runs with one or two passes. The full model card may contain additional information, but it was not included in the source material provided for this article.
Model Card and Local Tests
The next useful evidence would be a full model card with weight sizes, hardware guidance, benchmark scores and evaluation methods. Reproducible third-party tests could then compare quality, memory use and generation speed across supported runtimes and clearly specified devices.
For users considering installation, the practical next step is to check the model’s current documentation and runtime compatibility before downloading it. Testing the fixed three-pass model alongside the one- or two-pass research settings could clarify how speed and output quality change. Until those results are available, the release establishes availability and claimed capabilities, not verified performance on a given local setup.
Key Questions
What is Inkling 2 Mini?
Inkling 2 Mini is a multimodal mixture-of-experts model released by Thinking Machines Lab. The company describes it as accepting text and images, with about 26 billion total parameters and about 3 billion active per token.
Which local inference tools are listed?
The fixed-pass model is listed for Transformers, llama.cpp and MLX. The release also lists SGLang support for datacenter inference deployments.
Can users change how many layer-loop passes it uses?
The separate TML-d24-varloop-research checkpoint allows the number of passes to be changed. The company says fewer passes may improve speed but could reduce quality; the supplied material does not provide a full evaluation of that trade-off.
What hardware is required to run Inkling 2 Mini?
The source material does not state a minimum hardware configuration or complete model file size. Users should consult the current model documentation and test compatibility with their specific hardware.
Are the reported benchmark results independently verified?
Not in the supplied material. The results are attributed to Thinking Machines Lab, and the material does not include the full score table or independent replication.
Primary source: Hugging Face · via ThorstenMeyerAI.com
