Implementations
What you can actually run. Repository statistics verified September 2026.
Latent reasoning is research code, with one exception. The method-specific repositories are self-contained academic releases at GPT-2 to 8B scale, without packaging and frequently pinned to a stale dependency snapshot. Most reproduce their paper's numbers. None is something to add to a serving stack.
Open model checkpoints
Two families of open weights exist that reason by depth recurrence.
| Model | Scale | Notes | License |
|---|---|---|---|
| Ouro | 1.4B / 2.6B 7.7T tokens |
Looped language model, 24 layers, 4 recurrent steps by default with configurable early exit. Instruction-tuned -Thinking variants published alongside. Loads through standard Hugging Face tooling. Claims 3–4B dense-equivalent quality at 2.6B parameters. The only release here a practitioner could plausibly deploy rather than study. | Apache-2.0 |
| Huginn-0125 | 3.5B 800B tokens |
Prelude / recurrent core / coda, with user-controlled test-time recursion
(num_steps, typically 4–64) and entropy, KL and argmax-stability
early-exit criteria. Eight intermediate training checkpoints also published. Explicitly
a proof of concept and undertrained relative to modern dense models of the same size. |
Apache-2.0 |
Method implementations
| Repository | Method | Stars / last push | State |
|---|---|---|---|
| facebookresearch/coconut | Coconut: continuous thoughts arXiv:2412.06769 |
1,705 · 2026-07 | Training code and data for the GPT-2-scale GSM8K / ProsQA / ProntoQA experiments. No official checkpoints; community reproductions exist on the Hub. MIT. |
| seal-rg/recurrent-pretraining | Recurrent depth (Huginn) arXiv:2502.05171 |
961 · 2025-12 | Full pretraining stack behind Huginn-0125. Well documented, actively maintained. Apache-2.0. |
| raymin0223/mixture_of_recursions | Mixture-of-Recursions arXiv:2507.10524 |
626 · 2025-09 | Complete pretraining framework with per-token depth routing and matched KV caching. The most production-adjacent of the architecture-level methods. Apache-2.0. |
| ezelikman/quiet-star | Quiet-STaR arXiv:2403.09629 |
740 · 2024-08 | Stale. Patches a specific transformers dev snapshot from 2024 into
Mistral's modeling code; brittle to run against current versions. Apache-2.0. |
| InternLM/SIM-CoT | Supervised implicit CoT arXiv:2509.20317 |
215 · 2026-04 | The best-resourced and most current implicit-CoT repository. Reports gains over CoT, Coconut and CODI on GPT-2 and Llama 1B/3B/8B. Apache-2.0. |
| da03/Internalize_CoT_Step_by_Step | Stepwise internalisation arXiv:2405.14838 |
208 · 2025-04 | Ships data (including GSM8K-Aug), pretrained checkpoints and eval scripts. Genuinely reproduces the paper. MIT. |
| zjunlp/LightThinker | Gist-token compression arXiv:2502.15589 |
165 · 2026-06 | Actively maintained. MIT. |
| zhenyi4/codi | CODI: self-distillation arXiv:2502.21074 |
107 · 2025-12 | Self-contained GPT-2-scale training and eval; reproduces the GSM8K numbers. No license file: check before reuse. |
| xuyige/SoftCoT | SoftCoT and SoftCoT++ arXiv:2502.12134, 2505.11484 |
95 · 2025-05 | Training scripts, GSM8K preprocessing, checkpoints. No explicit license despite being public: check before reuse. |
| JacobPfau/fillerTokens | Filler tokens arXiv:2404.15758 |
76 · 2024-04 | Minimal research code for two synthetic algorithmic tasks. Dormant. No license. |
| mcleish7/retrofitting-recurrence | Retrofitted recurrence arXiv:2511.07384 |
69 · 2025-11 | Converts a pretrained non-recurrent model to depth-recurrent via curriculum. Apache-2.0. |
| yingfan-bot/lotus | LOTUS: supervised loop latents arXiv:2606.31779 |
51 · 2026-07 | Fine-tuning scripts over Llama-3.2-3B. no standalone checkpoint. MIT. |
| alperengozeten/CoT2 | Continuous CoT / parallel search arXiv:2505.23648 |
16 · 2026-02 | Accompanies the theory paper. MIT. |
| UW-Madison-Lee-Lab/looped-tf | Looped transformers, length generalisation arXiv:2409.15647 |
14 · 2025-04 | Small research codebase. |
Papers with no public implementation
- Pause tokens (Goyal et al., arXiv:2310.02226): never released by Google Research. The only related public repo is an independent toy that predates the paper and does not reproduce its pretraining setup.
- Compressed Chain of Thought (Cheng & Van Durme, arXiv:2412.13171): no release found anywhere. An unrelated multimodal-prompting repository shares the CCoT acronym.
- CoTFormer (arXiv:2310.10845): no official repository; only a later independent reproduction paper.
- Loop the Loopies! (arXiv:2607.16051): the largest looped model reported, with no public code or checkpoint.
Benchmarks and data
There is no shared benchmark for this field
ProsQA, a synthetic DAG-shaped multi-hop task with roughly 17.9k train and 300 validation examples at 3–6 hops, is the only benchmark purpose-built for latent reasoning, and it ships as JSON inside the Coconut repository rather than as an independent dataset. Everything else is borrowed: ProntoQA (synthetic first-order-logic QA, predating the field), GSM8K-Aug (385k augmented GSM8K samples with intermediate expressions stripped, now the de facto training corpus for implicit CoT), GSM-Hard, and the standard suites — MATH-500, AIME, SVAMP, BIG-Bench Hard, CommonsenseQA, ARC-C, GPQA.
None of these was designed to isolate whether a model is genuinely computing latently rather than taking a shortcut. Every critique paper in this area has had to build its own diagnostic. Cross-paper comparison is correspondingly unreliable, and this is a real, actionable gap rather than a search failure.
Framework support
None. No evidence that Transformers, vLLM, TGI, SGLang or llama.cpp exposes continuous-thought
decoding, recurrent-depth loop control, or pause-token mechanisms as a documented first-class
API, the way they expose KV-cache quantisation or speculative decoding. Huginn and Ouro ship
custom modeling files loaded through trust_remote_code=True, which is a packaging
convenience rather than a library feature. The absence is itself informative: every method
here remains a bespoke one-off.
Curated lists
-
Awesome-Latent-Space —
970 stars, last updated July 2026. The broadest and most-starred. (Formerly published under
the
YU-deepaccount. That path now redirects.) - EIT-NLP/Awesome-Latent-CoT — 382 stars, June 2026. Companion to the "Reasoning Beyond Language" survey and the most focused on latent chain of thought specifically.
- LatentCoT-Horizon — 414 stars, November 2025.