Families of methods

1. Internalisation by curriculum token → nothing

One keeps the vocabulary, and in the simplest version the architecture, and trains the explicit rationale away. A model is first taught to emit a chain of thought, then the intermediate tokens are progressively deleted during fine-tuning until none remain and the model answers from the question alone.

Deng et al. (2023) distil a teacher's horizontal, token-by-token reasoning into vertical, layer-by-layer computation, but not within one unchanged model: a separate emulator predicts the teacher's hidden states and a student answers from them, so inference runs two stacks. After the final joint fine-tuning, the intermediate states no longer decode to anything resembling the teacher's steps. The follow-up, stepwise internalisation (Deng, Choi & Shieber 2024), drops the emulator, keeps the base architecture, and simply removes tokens on a schedule. It takes GPT-2 Small to 99% on nine-by-nine multiplication, a task the same model cannot learn to do in one pass without the curriculum, and Mistral-7B past 50% on GSM8K with nothing emitted between question and answer.

This family is the cleanest demonstration that a fixed-depth forward pass has more capacity than standard training extracts from it. It is also the family where the curriculum does the most work, and where it is least clear the curriculum transfers.

2. Continuous thoughts token → vector

One deletes the discretisation, feeding the last hidden state back as the next input embedding. Coconut (Hao et al. 2024) is the reference implementation and the paper that named the idea.

The result that made the family interesting is not the headline accuracy but the mechanism. On ProsQA, a synthetic DAG-shaped multi-hop task released with the paper, Coconut reaches 97.0% against 77.5% for explicit chain of thought on a GPT-2 base, and the continuous thought vectors turn out to encode a frontier of several candidate reasoning paths rather than a single committed one, giving breadth-first-like search. Zhu et al. (2025) construct a two-layer transformer that uses $D$ continuous steps to solve directed graph reachability, where $D$ is the graph diameter, for a fixed maximum input length and with width growing with the vocabulary, one orthonormal token per node. The best known constant-depth discrete-CoT construction needs $O(n^2)$ decoding steps, and no lower bound is proved. In their trained two-layer models the frontier shows up in attention and embedding similarity, with no intervention testing whether the answer uses it.

The GSM8K result

On GSM8K, Coconut gets 34.1% against 42.9% for explicit chain of thought. The gains are concentrated on search and planning. On arithmetic the method loses.

No account of this family is honest without both numbers, and the paper adds a third qualification of its own. Run on Llama-3.2-3B and Llama-3-8B it still beats a no-CoT baseline, 31.7 against 26.0 and 43.6 against 42.2, but the authors note these gains are “not as pronounced as those previously demonstrated with GPT-2” and offer as “one possible reason” that larger models have already undergone extensive language-focused pre-training, making the transition to latent reasoning more challenging. The headline numbers are a small-model result, and the larger-model comparison is against emitting nothing rather than against explicit chain of thought.

Later work mostly addresses training rather than capability, and the training page covers it. CODI (Shen et al. 2025) replaces Coconut's multi-stage curriculum with self-distillation that aligns a single token's hidden state between an explicit and an implicit pass, and is the first in the family to match explicit CoT on GSM8K at GPT-2 scale.

SoftCoT (Xu et al. 2025) never fine-tunes the target model at all: a small frozen assistant produces soft thoughts that a trained projector maps into the frozen LLM's embedding space. The authors motivate this by catastrophic forgetting, which they conjecture from LoRA and LoRA-adapted Coconut baselines scoring below zero-shot CoT on instruction-tuned models. No experiment isolates forgetting as the cause. SIM-CoT (Wei et al. 2025) adds a step-level auxiliary decoder, discarded at inference, against the collapse into near-identical latent vectors that it observes in some failed runs.

3. Compression into latent tokens many tokens → few vectors

Rather than removing the intermediate representation, one shrinks it. A rationale that would have taken hundreds of tokens is encoded into a handful of dense vectors, called contemplation tokens, gist tokens, or thinking tokens depending on the paper, and the original text is discarded.

CCoT (Cheng & Van Durme 2024) fine-tunes an off-the-shelf decoder to generate variable-length dense representations in place of the rationale, plus a separate answer decoder, since the compressed states are out of distribution for the base model. The compression ratio is fixed at training time, with a learned stopping rule setting the count per example. At a ratio of 0.10 it buys roughly nine accuracy points over emitting nothing while adding about 0.4s of generation against 8.1s for the full decode.

LightThinker (Zhang et al. 2025) compresses during generation and evicts the original tokens from context using specialised attention masks. Heima (Shen et al. 2025) does the multimodal version and pairs it with trained interpreters that write approximate text back from the question and the thinking token, at a reasoning-stage BLEU of 11.2. KaVa (Kuzina et al. 2025) attacks the KV cache directly, distilling a compressed teacher cache into a latent student and finding CoT-bearing caches highly redundant.

The interpreters in Heima and the auxiliary decoder in SIM-CoT concede something: the compression loses the auditable trace, and the authors build machinery to buy some of it back.

4. Recurrent depth loop inside the stack

One spends the extra computation in depth rather than in sequence length. A block of layers is applied repeatedly to the same state, so test-time compute decouples from context length and from the vocabulary entirely. The antecedents are old: adaptive computation time (Graves 2016), Universal Transformers (Dehghani et al. 2018), PonderNet (Banino et al. 2021), but the modern instances are language models trained this way from scratch.

Geiping et al. (2025) pretrained Huginn-0125: 3.5B parameters, 800B tokens, a prelude/recurrent-core/coda design whose core is unrolled between 4 and 64 times at test time, with no chain-of-thought training data at all. Ouro (Zhu et al. 2025) is the largest confirmed effort, at 1.4B and 2.6B parameters over 7.7T tokens, and reports that the gains come from better knowledge manipulation rather than added knowledge capacity.

Mixture-of-Recursions (Bae et al. 2025) routes each token to its own recursion depth. McLeish et al. (2025) show recurrence can be retrofitted onto an already-pretrained model by a depth curriculum rather than requiring a fresh pretrain.

Saunshi et al. (2025) give the clearest statement of why this might work: a $k$-layer transformer looped $L$ times comes close to a non-looped $kL$-layer model on the reasoning tasks they measure, which points to many reasoning problems being bound by depth rather than by parameters.

5. Contentless tokens extra compute, no channel

The control experiment for the whole field. If intermediate tokens help because they carry reasoning content, then meaningless tokens should not help. If they help because they buy serial positions and computation, meaningless tokens should help too.

They help, conditionally. Pfau, Merrill & Bowman (2024) show transformers solving two algorithmic tasks using nothing but repeated dots where the same models do not solve them without the fillers, and give a quantifier-depth condition under which the benefit is available. Learning to exploit fillers requires dense, task-specific supervision and does not emerge from ordinary training. Goyal et al. (2024) get real downstream gains from learnable pause tokens on a 1B model, taking SQuAD exact match from 36.4 to 55.9, when pause tokens appear in both pretraining and fine-tuning. Adding them at fine-tuning alone gives mixed, smaller gains.

The negative result is equally important: Vennam et al. (2024) find naive thinking tokens underperform supervised chain of thought on small models. They hypothesise that the single repeated embedding receives noisy gradients, though they report no task accuracy for the two-embedding variant that would test it. Contentless tokens are a proof about expressivity, not a practical substitute.

The pause need not persist, or even occupy a position. Monea et al. (2026) interleave a learned prediction slot with each input token, apply the loss only at the prediction slot, and evict its keys and values after a short window, so the persistent cache holds only the input stream. From 53M to 1.678B parameters on FineWeb-Edu this lowers validation loss at every scale and adds 2.3 to 3.1 points to a five-task zero-shot average, at roughly double the training compute per step. The free pause tokens of Langford et al. (2026) drop the window: the prediction stream starts from one shared learned vector, shares the backbone weights, queries the input stream's keys and values at every layer and writes none of its own, adding no position, cache entry or decode step. On one 1B model trained on 100B tokens the gain is 0.028 nats of cross-entropy at equal tokens and 0.005 to 0.013 at equal node-hours, and individual downstream tasks stay within their 95% confidence intervals. Both papers measure language modelling and aggregate benchmark scores rather than chain-of-thought reasoning, and the free pause computation runs in parallel, layer by layer, adding no serial depth.

6. Hybrids and interleaving both channels

The pragmatic response to the fact that neither endpoint dominates. Token Assorted (Su et al. 2025) abstracts only the early portion of a trace into discrete VQ-VAE latent tokens and mixes them with ordinary text. Averaged over seven maths benchmarks, accuracy rises 3.7, 2.9 and 4.5 points over the best baseline at 1B, 3B and 8B, with a 13.3-point gain on one 8B task, and responses average about 17% fewer tokens than chain of thought. SpiralThinker (Piao & Park 2025) alternates latent iterations with text. LOTUS (Fan et al. 2026) supervises loop latents in parallel and is reported as the first latent-CoT method to close the gap with explicit CoT at 3B scale, having noted that earlier latent methods fall behind past 1B and that the gap widens with scale.

This is where the dial in the central diagram is actually being turned, and it is thinly populated compared with the two endpoints.

7. Tiny recursive networks recursion without an LLM

The other families all start from a language model. This one does not, and it reaches the same place from the opposite direction.

The Hierarchical Reasoning Model (Wang et al. 2025) uses two small recurrent modules running at different frequencies, a slow one for planning and a fast one for detail, and reports beating chain-of-thought LLMs on Sudoku, maze and ARC-AGI-style puzzles at 27M parameters trained on roughly a thousand examples. Jolicoeur-Martineau (2025) then stripped it further: one tiny recursive network, a simpler recipe, and reported results beating both HRM and much larger models on the same tasks.

Two things make this relevant here rather than a separate curiosity. The mechanism is recursion in a continuous state with no emitted intermediate tokens, matching the definition above. And the scale sits far below the recurrent-depth language models: HRM's 27M parameters are about 96× smaller than Ouro at 2.6B and 130× smaller than Huginn at 3.5B, and TRM's 7M about 370–500× smaller, two to nearly three orders of magnitude depending on which pair one takes.

Those headline counts are the recurrent network alone. On ARC both models also carry a per-puzzle embedding table, roughly 876,000 puzzles by 512 dimensions in TRM's case, registered as a buffer and so excluded from the reported figure. Whether it belongs in the count is disputed in both projects' issue trackers and unanswered by their maintainers.

The systems differ in architecture, training regime, data and task, so the ratio does not isolate depth as the cause. What it supports is narrower: small recurrent networks report strong results on these puzzle benchmarks.

Several results cut against the reading. Ren and Liu (2026) find HRM can fail on a Sudoku with one unknown cell, because a correct answer is not always a stable fixed point, and that it tends to settle on the first fixed point it reaches, sometimes a wrong one. They call this “guessing” rather than reasoning. They do not ablate the hierarchy itself. The team that runs the benchmark did: in their own ablations the hierarchical architecture “had minimal performance impact when compared to a similarly sized transformer,” while the under-documented outer refinement loop drove most of the gain, and “most of the performance comes from memorizing solutions to the specific tasks used at evaluation time.” They also note that the model can only be applied to puzzle identifiers seen during training, so the inference data has to be part of the training set.

The per-puzzle identifier carries more of the result than the architecture description suggests. Roye-Azar et al. (2025) report that replacing it with a blank or random token drops ARC accuracy to zero, and an independent ablation in the project's issue tracker reproduces that figure. The architecture works on the benchmarks. What it is doing to get there is contested.