Families of methods
Six ways to move reasoning off the transcript, and what each one is actually doing.
Methods differ along two independent dimensions: where the reasoning loop closes, and what crosses from one step to the next. The named families cut across both, and several methods are hybrids by construction.
1. Internalisation by curriculum token → nothing
One keeps the architecture and the vocabulary exactly as they are, and trains the explicit rationale away. A model is first taught to emit a chain of thought, then the intermediate tokens are progressively deleted during fine-tuning until none remain and the same forward pass must do the work that used to be spread over many.
Deng et al. (2023) do this by distilling a teacher's horizontal, token-by-token reasoning into a student's vertical, layer-by-layer computation. The follow-up, stepwise internalisation (Deng, Choi & Shieber 2024), drops the emulator and simply removes tokens on a schedule. It takes GPT-2 Small to 99% on nine-by-nine multiplication, a task the same model cannot learn to do in one pass without the curriculum, and Mistral-7B past 50% on GSM8K with nothing emitted between question and answer.
This family is the cleanest demonstration that a fixed-depth forward pass has more capacity than standard training extracts from it. It is also the family where the curriculum does the most work, and where it is least clear the curriculum transfers.
2. Continuous thoughts token → vector
One deletes the discretisation, feeding the last hidden state back as the next input embedding. Coconut (Hao et al. 2024) is the reference implementation and the paper that named the idea.
The result that made the family interesting is not the headline accuracy but the mechanism. On ProsQA, a synthetic DAG-shaped multi-hop task released with the paper, Coconut reaches 97.0% against 77.5% for explicit chain of thought at GPT-2 scale, and the continuous thought vectors turn out to encode a frontier of several candidate reasoning paths rather than a single committed one, giving breadth-first-like search. Zhu et al. (2025) proved this is not an artifact: a two-layer transformer using $D$ continuous steps solves directed graph reachability where $D$ is the graph diameter, while the best known discrete construction needs $O(n^2)$ decoding steps.
The same paper's other number matters as much
On GSM8K, Coconut gets 34.1% against 42.9% for explicit chain of thought. The gains are concentrated on search and planning. On arithmetic the method loses.
No account of this family is honest without both numbers.
Later work mostly addresses training rather than capability. CODI (Shen et al. 2025) replaces Coconut's multi-stage curriculum with self-distillation that aligns a single token's hidden state between an explicit and an implicit pass, and is the first in the family to match explicit CoT on GSM8K at GPT-2 scale.
SoftCoT (Xu et al. 2025) never fine-tunes the target model at all: a small frozen assistant produces soft thoughts that a trained projector maps into the frozen LLM's embedding space, which is an explicit workaround for the catastrophic forgetting that full-parameter latent fine-tuning causes on models that already reason well. SIM-CoT (Wei et al. 2025) adds a step-level auxiliary decoder, discarded at inference, to stop latent vectors homogenising.
3. Compression into latent tokens many tokens → few vectors
Rather than removing the intermediate representation, shrink it. A rationale that would have taken hundreds of tokens is encoded into a handful of dense vectors, called contemplation tokens, gist tokens, or thinking tokens depending on the paper, and the original text is discarded.
CCoT (Cheng & Van Durme 2024) generates variable-length dense representations as a drop-in replacement for the rationale on off-the-shelf decoders, with a tunable compression ratio. At a ratio of 0.10 it buys roughly nine accuracy points over emitting nothing while adding about 0.4s of generation against 8.1s for the full decode.
LightThinker (Zhang et al. 2025) compresses during generation and evicts the original tokens from context using specialised attention masks. Heima (Shen et al. 2025) does the multimodal version and pairs it with an auxiliary interpreter that reconstructs the discarded text. KaVa (Kuzina et al. 2025) attacks the KV cache directly, distilling a compressed teacher cache into a latent student and finding CoT-bearing caches highly redundant.
The interpreters in Heima and the auxiliary decoder in SIM-CoT concede something: the compression loses the auditable trace, and the authors build machinery to buy some of it back.
4. Recurrent depth loop inside the stack
One spends the extra computation in depth rather than in sequence length. A block of layers is applied repeatedly to the same state, so test-time compute decouples from context length and from the vocabulary entirely. The antecedents are old: adaptive computation time (Graves 2016), Universal Transformers (Dehghani et al. 2018), PonderNet (Banino et al. 2021), but the modern instances are language models trained this way from scratch.
Geiping et al. (2025) pretrained Huginn-0125: 3.5B parameters, 800B tokens, a prelude/recurrent-core/coda design whose core is unrolled between 4 and 64 times at test time, with no chain-of-thought training data at all. Ouro (Zhu et al. 2025) is the largest confirmed effort, at 1.4B and 2.6B parameters over 7.7T tokens, and reports that the gains come from better knowledge manipulation rather than added knowledge capacity.
Mixture-of-Recursions (Bae et al. 2025) routes each token to its own recursion depth. McLeish et al. (2025) show recurrence can be retrofitted onto an already-pretrained model by a depth curriculum rather than requiring a fresh pretrain.
Reddi et al. (2025) give the cleanest statement of why this works: a $k$-layer transformer looped $L$ times nearly matches a non-looped $kL$-layer model on reasoning tasks, so many reasoning problems are bound by depth rather than by parameters.
5. Contentless tokens extra compute, no channel
The control experiment for the whole field. If intermediate tokens help because they carry reasoning content, then meaningless tokens should not help; if they help because they buy serial positions and computation, meaningless tokens should help too.
They help, conditionally. Pfau, Merrill & Bowman (2024) show transformers solve two algorithmic tasks unsolvable in a single pass using nothing but repeated dots, and characterise which problems admit the benefit via the quantifier depth of a first-order description, but learning to exploit fillers requires dense, task-specific supervision and does not emerge from ordinary training. Goyal et al. (2024) get real downstream gains from learnable pause tokens on a 1B model, including +18% exact match on SQuAD, but only when pause tokens appear in both pretraining and fine-tuning.
The negative result is equally important: Vennam et al. (2024) find naive thinking tokens consistently underperform real chain of thought, because a single repeated embedding produces noisy gradients. Contentless tokens are a proof about expressivity, not a practical substitute.
6. Hybrids and interleaving both channels
The pragmatic response to the fact that neither endpoint dominates. Token Assorted (Su et al. 2025) abstracts only the early portion of a trace into discrete VQ-VAE latent tokens and mixes them with ordinary text, reporting gains of 4–13% across model sizes with traces about 17% shorter. SpiralThinker (Piao & Park 2025) alternates latent iterations with text. LOTUS (Fan et al. 2026) supervises loop latents in parallel and is reported as the first latent-CoT method to close the gap with explicit CoT at 3B scale, having noted that earlier latent methods fall behind past 1B and that the gap widens with scale.
This is where the dial in the central diagram is actually being turned, and it is thinly populated compared with the two endpoints.
Failure modes
Latent representations collapse as you add steps
SIM-CoT documents this as latent instability: as the number of latent reasoning tokens grows, the vectors homogenise, lose semantic diversity, and training collapses. That the fix is step-level auxiliary supervision is itself evidence that end-to-end latent training under-constrains the representation.
The superposition story does not hold everywhere
Soft Thinking (Zhang et al. 2025) replaces each token with a probability-weighted mixture of embeddings, claiming parallel exploration. Wu et al. (2025) showed directly that the model overwhelmingly follows the single highest-probability component: the method degenerates to greedy decoding unless noise is deliberately injected. Rizvi-Martel, Rabusseau & Mosbach (2026) sharpen this: genuine superposition shows up in models trained from scratch with continuous latent CoT, and largely collapses under the fine-tuning regimes most practitioners actually use. Superposition is a per-method property to be demonstrated, not a family-wide entitlement.
More depth is not monotonically better
Yang et al. (2026) document looped language model performance peaking at some recurrence depth and then collapsing with further looping, and propose spectral-radius regularisation as a fix. This bears directly on the "infinite depth" framing several surveys use.
Prompted implicit reasoning is mostly shortcuts
Yu (2024) finds that models merely prompted to reason implicitly barely compute genuine intermediate steps and lean on memorised shortcuts; models specifically trained for implicit CoT do show evidence of real internal stepwise computation. Both are brittle to superficial reformatting. Claims that "the model is reasoning internally" need to say which of these two situations they are in.
Accuracy may be a poor proxy, but the measurement is contested
Sahoo et al. (2026) report that on Qwen2.5-Math-7B at 61% accuracy, 18.4% of correct predictions use stable faithful reasoning while 81.6% “emerge through computationally inconsistent pathways,” with 8.8% silent failures and a weak negative correlation ($r = -0.21$) between reasoning quality and correctness.
Those figures should not be read as established failure rates, and this page previously presented them that way. Three problems, raised by a reader and checked here:
- The reported statistics are mutually inconsistent. A negative correlation with binary correctness cannot coexist with a higher mean fidelity for correct examples than for incorrect ones — $\mathrm{Cov}(F,Y) = p(1-p)\{E[F \mid Y{=}1] - E[F \mid Y{=}0]\}$ is positive whenever the conditional means are ordered that way. Something has been mislabelled, and which it is has not been established.
- The headline rates are definitions built on proxies, not independent verification of reasoning errors. “Confident” is an activation-consistency score above a threshold, not a calibrated output probability, so a correct answer falling below that threshold has not thereby been shown to be a guess.
- Some of the underlying metrics are partly fixed by construction: a statistic defined as the fraction of layers above their own median sits near one half whatever the model does.
The paper is worth keeping as a contested diagnostic proposal and a reminder that accuracy alone tells you little. It is one model family and a few hundred GSM8K items, and it is not evidence about depth-recurrent or continuous-feedback architectures generally.
The evidence base is small and synthetic
Much of the strongest support, graph reachability, ProsQA, ProntoQA, maze tasks, comes from benchmarks built to exhibit parallel search. Most strong empirical results are at GPT-2 to 8B scale. Cross-paper comparison is largely apples-to-oranges, for the reasons given under implementations.