
Inkling-2-Mini-Infall_TT

What is Inkling-2-Mini-Infall_TT?
Inkling-2-Mini-Infall_TT is an 8-bit GGUF quantization of thinkingmachines/Inkling-2-Mini, released by Thinking Machines Lab. Inkling-2-Mini is a multimodal autoregressive looped transformer with 26.4B total parameters and 3.4B active per token, built as a mixture of experts. It accepts images and text, and supports reasoning and tool use.
The GGUF files are standard and run on any machine with llama.cpp. We publish seven files, from 18.7 GB (Q4_K_M) to 27.8 GB (Q8_0), so you can trade file size against fidelity to fit your memory.
Specification
| Original model | thinkingmachines/Inkling-2-Mini |
|---|---|
| Model provider | Thinking Machines Lab, Inc. |
| Date of release | November 20, 2026 |
| Languages | English, general multilingual support |
| Model type | Multimodal autoregressive looped transformer |
| Parameters | 26.4B total, 3.4B active |
| Context length | 1,048,576 tokens (GGUF metadata) |
| Format | GGUF, importance-matrix quantization |
| License | Apache 2.0 |
| Quantized with | Infall-TT (Synthetic Minds), for Tenstorrent hardware |
| Developer | Synthetic Minds |
| Target hardware | Tenstorrent Blackhole P150A/B, P300C, Galaxy Blackhole, or any machine running llama.cpp |
Available quantizations
Each file is a single GGUF. File sizes are the exact sizes on Hugging Face. Quality figures are measured against the Q8_0 reference (see below).
| Quant | Size | Top-1 vs Q8_0 | Mean KL | File |
|---|---|---|---|---|
| Q4_K_MSmallest | 18.7 GB | ~96.8% | ~0.0059 | Download |
| Q5_K_S | 19.2 GB | ~97.3% | ~0.0041 | Download |
| Q5_K_M | 21.2 GB | ~97.5% | ~0.0037 | Download |
| Q6_K_S | 24.0 GB | ~97.7% | ~0.0028 | Download |
| Q6_K | 24.7 GB | ~97.8% | ~0.0027 | Download |
| Q6_K_L | 25.4 GB | ~97.9% | ~0.0026 | Download |
| Q8_0Reference | 27.8 GB | 100% | 0 | Download |
Quantization quality
We compared every quant against the Q8_0 reference on wikitext-2. Smaller files cost a little fidelity, but every quant stays close to Q8_0. The numbers are approximate values read from the charts.
Output distortion (KL divergence to Q8_0)

Distortion drops fastest between Q4_K_M and Q5_K_S. From Q6_K_S upward the gains are small, with Q6_K_L only marginally better than Q6_K_S.
Top-1 token agreement

Even the smallest quant, Q4_K_M, makes the same next-token choice as Q8_0 about 97 times in 100.
Perplexity

Perplexity differences between all quants (about 0.01) are far smaller than the error bars (about ±0.035), so on this measure they are statistically indistinguishable from Q8_0.
| Quant | Size (GB) | Mean KL | Top-1 | Perplexity |
|---|---|---|---|---|
| Q4_K_M | 18.7 | ~0.0059 | ~96.8% | ~5.943 |
| Q5_K_S | 19.2 | ~0.0041 | ~97.3% | ~5.936 |
| Q5_K_M | 21.2 | ~0.0037 | ~97.5% | ~5.935 |
| Q6_K_S | 24.0 | ~0.0028 | ~97.7% | ~5.931 |
| Q6_K | 24.7 | ~0.0027 | ~97.8% | ~5.934 |
| Q6_K_L | 25.4 | ~0.0026 | ~97.9% | ~5.933 |
| Q8_0 | 27.8 | 0 (reference) | 100% (reference) | ~5.931 |
Download
Using the Hugging Face CLI:
hf download synthetic-minds/Inkling-2-Mini-Infall_TT-GGUF --include "Inkling-2-Mini-Infall_TT-Q8_0.gguf" --local-dir ./Change the file name in --include to download a different quantization.
Run with llama.cpp
Install llama.cpp on macOS or Linux:
curl -LsSf https://llama.app/install.sh | shOr on Windows (PowerShell):
irm https://llama.app/install.ps1 | iexThen start a server straight from the Hugging Face repo:
llama-server -hf synthetic-minds/Inkling-2-Mini-Infall_TT-GGUF:Q8_0Or run the file you downloaded:
llama-server -m ./Inkling-2-Mini-Infall_TT-Q8_0.ggufRun with Ollama
Install Ollama on macOS or Linux:
curl -fsSL https://ollama.com/install.sh | shOn Windows, use the installer from ollama.com/download. Then run the model directly from the repo:
ollama run hf.co/synthetic-minds/Inkling-2-Mini-Infall_TT-GGUF:Q8_0Evaluations
Selected results from the original model card, as reported by Thinking Machines Lab. They were measured on the original model, not on this quantization, so results from the 8-bit GGUF may differ slightly.
| Inkling 2 Mini | Gemma 4 31B | Muse Glimmer | Qwen3.8-27B | Nemotron 3.5 Super VL | |
|---|---|---|---|---|---|
| Params (B) activated / total | 5/26 | 31/31 | 30/30 | 27/27 | 12/125 |
| Overall | |||||
| AA Index (v5.0) | 20.1 | 4.4 | 6.3 | 20.1 | 13.6 |
| Coding | |||||
| SWEBench Verified (bash only) | 79.2% | 46.1% | 76.0% | 77.8% | 71.1% |
| SWEBench Pro v2 (Hard) | 77.9% | 14.2% | 44.3% | 73.8% | 44.1% |
| Terminal Bench v2.1 | 71.6% | 43.4% | 51.7% | 73.0% | 60.7% |
| Agentic search & tool use | |||||
| BrowseComp | 79.0% | 37.0% | 63.9% | 61.6% | 78.9% |
| DeepSearchQA | 85.3% | 61.7% | 74.6% | 79.7% | 77.7% |
| MCP Atlas | 82.7% | 54.2% | 75.5% | 77.3% | 67.7% |
| Vision | |||||
| MMMU Pro (Standard 10) | 74.9% | 73.4% | 74.3% | 76.3% | 72.3% |
| OmniDocBench v1.6 | 92.8% | 82.5% | 90.4% | 91.5% | 77.9% |
| ScreenSpot Pro (original / with Python) | 76.5 / 87.7% | 5.9% / – | 75.4% / – | 57.6 / 78.9% | 43.5% / – |
| Reasoning & long context | |||||
| HLE-Diamond | 15.2% | 7.2% | 9.1% | 15.3% | 8.8% |
| ARC-AGI (v1 / v2) | 84.5 / 49.3% | – | – | 87.5 / 42.4% | 22.7 / 1.5% |
| LongBench v2 | 66.9% | 52.9% | 33.1% | 52.3% | 62.1% |
| Safety | |||||
| FORTRESS (adversarial) | 89.5% | 67.8% | 90.3% | 78.8% | 79.6% |
| Misbehavior (internal) ↓ | 9.8% | 27.6% | 46.4% | 57.7% | 52.8% |
Bold marks the best reported score in each row (lowest where ↓ is shown). See the original model card for the full set of benchmarks and methodology.
Frequently asked questions
What is Inkling-2-Mini-Infall_TT?
Inkling-2-Mini-Infall_TT is a set of GGUF quantizations of Inkling-2-Mini, a multimodal autoregressive looped transformer from Thinking Machines Lab with 26.4B total and 3.4B active parameters (mixture-of-experts). Synthetic Minds quantized it with the beta preview of Infall-TT. The files are standard GGUF and run on any machine with llama.cpp.
How do I run Inkling-2-Mini-Infall_TT locally?
Install llama.cpp and run llama-server -hf synthetic-minds/Inkling-2-Mini-Infall_TT-GGUF:Q8_0, or run ollama run hf.co/synthetic-minds/Inkling-2-Mini-Infall_TT-GGUF:Q8_0. You can also download a file with the Hugging Face CLI and start it with llama-server -m.
Which quantization should I download?
Q8_0 (27.8 GB) is the reference. Q4_K_M (18.7 GB) is the smallest and still picks the same next token as Q8_0 about 97 times in 100 on wikitext-2 (96.8%). Distortion drops fastest between Q4_K_M and Q5_K_S; from Q6_K_S upward the gains are small.
Does quantization hurt quality?
Smaller files cost a little fidelity, but every quant stays close to Q8_0. Perplexity differences between all quants are about 0.01, far smaller than the error bars of about ±0.035, so on that measure they are statistically indistinguishable from Q8_0. Values are approximate, read from the charts on the model card.
Do the benchmark results apply to these GGUF files?
No. The benchmark table comes from the original model card published by Thinking Machines Lab. It was measured on the original model, not on this quantization, so results from the 8-bit GGUF may differ slightly.
Can I use it commercially?
The original model is released by Thinking Machines Lab, Inc. under the Apache 2.0 license, and these quantized weights follow the same license.
Does it run on Tenstorrent hardware?
The target hardware is Tenstorrent Blackhole P150A/B, P300C and Galaxy Blackhole, or any machine running llama.cpp. The GGUF files themselves are standard and run on any machine with llama.cpp. Infall-TT is in beta preview and is not publicly available yet.
Support our work
If this quant is useful to you, you can support Synthetic Minds on Buy Me a Coffee, or join the Infall-TT waitlist and follow us on X and Hugging Face.
License
The original model is released by Thinking Machines Lab, Inc. under the Apache 2.0 license, and these quantized weights follow the same license. Copyright 2026 Synthetic Minds LTD.