Bibliography

arXiv identifiers were checked against the arXiv and Semantic Scholar APIs in September 2026. The same material is available as a map and a timeline.

The core

• Hao, S., Sukhbaatar, S., Su, D., Li, X., Hu, Z., Weston, J., and Tian, Y. (2024). Training Large Language Models to Reason in a Continuous Latent Space. arXiv:2412.06769. Coconut. Feeds the final hidden state back as the next input embedding, deleting the quantiser. ProsQA 97.0% against 77.5% for explicit chain of thought, and GSM8K 34.1% against 42.9%. The paper that named the idea and, read carefully, bounded it.
• Geiping, J., McLeish, S., Jain, N., Kirchenbauer, J., Singh, S., Bartoldson, B. R., Kailkhura, B., Bhatele, A., and Goldstein, T. (2025). Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach. arXiv:2502.05171. The other endpoint: 3.5B parameters over 800B tokens, a recurrent core unrolled 4 to 64 times at test time with no chain-of-thought training data. Open weights as Huginn-0125.
• Xu, K., and Sato, I. (2025). A Formal Comparison Between Chain of Thought and Latent Thought. arXiv:2509.25239. At $\log^k n$ iterations, polynomial width and constant precision, looped/continuous models are characterised exactly (Thm 3.12), while chain of thought is bounded above by $\mathsf{TC}^{k-1}$ (Lem 3.13). The strict separation (Thm 3.15) assumes $\mathsf{TC}^{k-1} \subsetneq \mathsf{TC}^k$. The countervailing stochastic result assumes $\mathsf{FPTAS} \subsetneq \mathsf{FPRAS}$ for self-reducible relations and compares against deterministic latent updates only.
• Korbak, T., Balesni, M., Barnes, E., Bengio, Y., Benton, J., et al. (2025). Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safety. arXiv:2507.11473. Forty-one authors across OpenAI, Anthropic, Google DeepMind, UK AISI, Redwood and Apollo. Argues current models are architecturally forced to externalise serial computation legibly, that this is an accident rather than a guarantee, and names fully latent reasoning as the threat to it.

Antecedents

• Graves, A. (2016). Adaptive Computation Time for Recurrent Neural Networks. arXiv:1603.08983. A differentiable halting mechanism: let the network choose how many internal steps to take. The ancestor of everything in the depth column.
• Dehghani, M., Gouws, S., Vinyals, O., Uszkoreit, J., and Kaiser, Ł. (2018). Universal Transformers. arXiv:1807.03819. Weight-tied layers plus per-position adaptive halting: depth becomes a loop. The architectural precursor to recurrent-depth language models.
• Banino, A., Balaguer, J., and Blundell, C. (2021). PonderNet: Learning to Ponder. arXiv:2107.05407. Replaces the halting penalty with a learned stochastic halting distribution under a proper probabilistic objective.
• Giannou, A., Rajput, S., Sohn, J., Lee, K., Lee, J. D., and Papailiopoulos, D. (2023). Looped Transformers as Programmable Computers. arXiv:2301.13196. A hand-programmed thirteen-layer transformer in a loop emulates a general-purpose computer. Looping as universality, independent of scale.

Expressivity theory

• Merrill, W., and Sabharwal, A. (2023). “The Parallelism Tradeoff: Limitations of Log-Precision Transformers.” TACL. arXiv:2207.00729. One forward pass of a log-precision transformer lies inside logspace-uniform $\mathsf{TC}^0$. Conditional on $\mathsf L \neq \mathsf P$, no polynomial-size family solves certain $\mathsf P$-complete problems in a single pass: an asymptotic statement about growing inputs, not a verdict on finite instances.
• Feng, G., Zhang, B., Gu, Y., Ye, H., He, D., and Wang, L. (2023). “Towards Revealing the Mystery behind Chain of Thought: A Theoretical Perspective.” NeurIPS. arXiv:2305.15408. Assuming $\mathsf{TC}^0 \neq \mathsf{NC}^1$, no fixed-depth log-precision transformer of polynomial width solves arithmetic-expression evaluation or linear-equation solving over $\mathbb{Z}_p$ at all sizes; constant-size autoregressive models with chain of thought can, and can implement dynamic programming. Conditional, and scoped to those two problem families: not “arithmetic” generally.
• Merrill, W., and Sabharwal, A. (2024). “The Expressive Power of Transformers with Chain of Thought.” ICLR. arXiv:2310.07923. Expressivity graded by chain-of-thought length: logarithmic is marginal, linear recognises all regular languages, polynomial is exactly $\mathsf P$. Each result carries a specific pre-norm architectural assumption.
• Li, Z., Liu, H., Zhou, D., and Ma, T. (2024). “Chain of Thought Empowers Transformers to Solve Inherently Serial Problems.” ICLR. arXiv:2402.12875. At constant precision and depth without chain of thought, expressivity is confined to $\mathsf{AC}^0$; $T$ steps of chain of thought simulate circuits of size $O(T)$. Tokens buy serial computation that a fixed-depth parallel circuit structurally cannot do.
• London, C., and Kanade, V. (2025). Pause Tokens Strictly Increase the Expressivity of Constant-Depth Transformers. arXiv:2505.21024. At constant precision and $O(\log n)$ width, polynomial pause-token padding characterises exactly $\mathsf{AC}^0$ (Thm 4.1), with a non-uniform strict separation from the unpadded case (Cor 4.2). The uniform separation needs quasi-polynomial padding (Thm 4.3), and log precision with polynomial padding gives $\mathsf{TC}^0$ (Thm 4.5). Padding buys parallel workspace at fixed depth, not serial depth. The PARITY result in §5 is a finite-size learning experiment requiring intermediate supervision, not the separation's witness.
• Zhu, H., Hao, S., Hu, Z., Jiao, J., Russell, S., and Tian, Y. (2025). Reasoning by Superposition: A Theoretical Perspective on Chain of Continuous Thought. arXiv:2505.12514. A two-layer transformer with $D$ continuous steps solves directed graph reachability at diameter $D$, where the best known discrete construction needs $O(n^2)$ steps. The superposition emerges from training rather than being built in. The companion paper on training dynamics is arXiv:2509.23365.
• Gozeten, H. A., Ildiz, M. E., Zhang, X., Harutyunyan, H., Rawat, A. S., and Oymak, S. (2025). Continuous Chain of Thought Enables Parallel Exploration and Reasoning. arXiv:2505.23648. Quantifies the parallelism as tracking several discrete traces at once, with a constructive one-layer solution to subset-sum.
• Reddi, S., Kumar, S., Li, Z., Dikkala, N., and Saunshi, N. (2025). Reasoning with Latent Thoughts: On the Power of Looped Transformers. arXiv:2502.17416. A $k$-layer transformer looped $L$ times nearly matches a $kL$-layer unlooped model: many reasoning problems are depth-bound rather than parameter-bound.
• Xu, K., and Sato, I. (2025). To CoT or To Loop? A Formal Comparison Between Chain-of-Thought and Looped Transformers. arXiv:2505.19245. Loops efficiently emulate parallel DAG-structured deterministic computation; sequential chain of thought suits self-reducible compositional problems. No universal conversion between loop count and token count: the relationship is structural.
• Zhang, H. (2026). Chain-of-Thought and Compressed Looped Transformers: A Memory-Budget Separation. arXiv:2605.30757. Under a standard complexity assumption, $\mathsf P \not\subseteq \mathsf{DSPACE}(\mathrm{polylog}\,n)$, a compressed-state loop carrying polylogarithmic persistent memory cannot decide problems $\mathsf P$-complete under logspace reductions that polynomial-length chain of thought can. Explicitly does not cover full sequence-state loops carrying $ndp$ bits. Recurrence buys depth, not memory.

Continuous and internalised chain of thought

• Deng, Y., Prasad, K., Fernandez, R., Smolensky, P., Chaudhary, V., and Shieber, S. (2023). Implicit Chain of Thought Reasoning via Knowledge Distillation. arXiv:2311.01460. Distils a teacher's horizontal token-by-token reasoning into a student's vertical layer-by-layer computation. The first serious attempt to delete the rationale and keep the capability.
• Zelikman, E., Harik, G., Shao, Y., Jayasiri, V., Haber, N., and Goodman, N. D. (2024). Quiet-STaR: Language Models Can Teach Themselves to Think Before Speaking. arXiv:2403.09629. Generates rationales after every token and rewards those that improve prediction of the real continuation. Zero-shot GSM8K 5.9% to 10.9%. The rationales are still tokens: a bridge case rather than latent reasoning proper.
• Deng, Y., Choi, Y., and Shieber, S. (2024). From Explicit CoT to Implicit CoT: Learning to Internalize CoT Step by Step. arXiv:2405.14838. Removes reasoning tokens on a schedule. GPT-2 Small reaches 99% on nine-by-nine multiplication; Mistral-7B passes 50% on GSM8K emitting nothing between question and answer.
• Cheng, J., and Van Durme, B. (2024). Compressed Chain of Thought: Efficient Reasoning Through Dense Representations. arXiv:2412.13171. Variable-length dense contemplation tokens as a drop-in for the rationale on off-the-shelf decoders, with a tunable compression ratio. No public implementation.
• Su, D., Zhu, H., Xu, Y., Jiao, J., Tian, Y., and Zheng, Q. (2025). Token Assorted: Mixing Latent and Text Tokens for Improved Language Model Reasoning. arXiv:2502.03275. Abstracts only the early part of a trace into discrete VQ-VAE latent tokens and mixes them with text. Gains of 4–13% across model sizes with traces about 17% shorter: a hybrid by construction.
• Xu, Y., Guo, X., Zeng, Z., and Miao, C. (2025). SoftCoT: Soft Chain-of-Thought for Efficient Reasoning with LLMs. arXiv:2502.12134. Never fine-tunes the target model: a frozen assistant produces soft thoughts that a trained projector maps into the frozen model's embedding space. An explicit dodge of the catastrophic forgetting that latent fine-tuning causes on strong reasoners. Extended for test-time scaling in arXiv:2505.11484.
• Shen, Z., Yan, H., Zhang, L., Hu, Z., Du, Y., and He, Y. (2025). CODI: Compressing Chain-of-Thought into Continuous Space via Self-Distillation. arXiv:2502.21074. Aligns one token's hidden state between an explicit and an implicit pass, removing Coconut's multi-stage curriculum. First in the family to match explicit chain of thought on GSM8K at GPT-2 scale.
• Zhang, J., Zhu, Y., Sun, M., Luo, Y., Qiao, S., et al. (2025). LightThinker: Thinking Step-by-Step Compression. arXiv:2502.15589. Compresses thoughts into gist tokens during generation and evicts the originals from context via specialised attention masks.
• Shen, X., Wang, Y., Zhou, Y., Shi, S., Zhao, X., et al. (2025). Efficient Reasoning with Hidden Thinking. arXiv:2501.19201. The Heima framework: multimodal reasoning condensed into a few abstract thinking tokens, with an information-theoretic analysis and an auxiliary interpreter that reconstructs the discarded text.
• Zhang, Z., He, X., Yan, T., Shen, Y., Zhao, C., et al. (2025). Soft Thinking: Unlocking the Reasoning Potential of LLMs in Continuous Concept Space. arXiv:2505.15778. Training-free: replace each token with a probability-weighted mixture of embeddings. One should read this alongside the rebuttal.
• Wei, X., Liu, Z., Zang, Y., Dong, X., Cao, Y., et al. (2025). SIM-CoT: Supervised Implicit Chain-of-Thought. arXiv:2509.20317. Diagnoses latent instability, latent vectors homogenise as steps are added until training collapses, and fixes it with a step-level auxiliary decoder removed at inference.
• Butt, N., Kwiatkowski, A., Labiad, I., Kempe, J., and Ollivier, Y. (2025). Soft Tokens, Hard Truths. arXiv:2509.19170. First scalable reinforcement learning of continuous chain of thought without distilling from discrete traces. Matches discrete chain of thought at pass@1 and exceeds it at pass@32 through greater trace diversity.
• Piao, S., and Park, S. (2025). SpiralThinker: Latent Reasoning through an Iterative Process with Text-Latent Interleaving. arXiv:2511.08983. Alternates latent iterations with text, with a progressive alignment objective to stabilise the latent updates.

Recurrent depth and looped models

• Mohtashami, A., Pagliardini, M., and Jaggi, M. (2023). CoTFormer: A Chain-of-Thought Driven Architecture with Budget-Adaptive Computation Cost at Inference. arXiv:2310.10845. Motivates looping explicitly as a token-level analogue of chain of thought, with per-token adaptive depth. No official implementation.
• Fan, Y., Du, Y., Ramchandran, K., and Lee, K. (2024). Looped Transformers for Length Generalization. arXiv:2409.15647. Task-adaptive iteration counts give near-perfect generalisation past forty-digit parity when trained only to twenty, where fixed-depth transformers fail outright.
• Bae, S., Fisch, A., Harutyunyan, H., Ji, Z., Kim, S., and Schuster, T. (2024). Relaxed Recursive Transformers: Effective Parameter Sharing with Layer-wise LoRA. arXiv:2410.20672. Converts pretrained transformers into looped ones, then relaxes the weight-tying with per-depth low-rank adapters.
• Bae, S., Kim, Y., Bayat, R., Kim, S., Ha, J., et al. (2025). Mixture-of-Recursions: Learning Dynamic Recursive Depths for Adaptive Token-Level Computation. arXiv:2507.10524. A router picks each token's recursion depth, with matched KV caching: a new Pareto frontier for perplexity per FLOP and for throughput.
• Zhu, R.-J., Wang, Z., Hua, K., Zhang, T., Li, Z., et al. (2025). Scaling Latent Reasoning via Looped Language Models. arXiv:2510.25741. Ouro. 1.4B and 2.6B looped models over 7.7T tokens, with recurrence built into pretraining rather than bolted on at test time. Attributes the gains to better knowledge manipulation rather than added capacity, and reports latent traces better aligned to final outputs than explicit chains. Open weights.
• McLeish, S., Li, X., Kirchenbauer, J., Kalra, D., Bartoldson, B. R., et al. (2025). Teaching Pretrained Language Models to Think Deeper with Retrofitted Recurrence. arXiv:2511.07384. Retrofits recurrence onto an already-pretrained model via a depth curriculum, decoupling test-time from training-time compute after the fact.
• Fan, Y., Svete, A., and Lee, K. (2026). Bridging the Gap Between Latent and Explicit Reasoning with Looped Transformers. arXiv:2606.31779. LOTUS. Notes that latent chain-of-thought methods had been falling behind explicit chain of thought past 1B with the gap widening at scale, and reports closing it at 3B via parallel supervision of loop latents, at 2.5–6.9× lower thought-phase latency.
• Blayney, T., Arroyo, D. M., Obando-Ceron, J., Castro, P. S., Courville, A., et al. (2026). A Mechanistic Analysis of Looped Reasoning Language Models. arXiv:2604.11791. Each layer in the loop converges toward a distinct fixed point and the block settles into a stable cyclic trajectory. Convergence is not automatic. It is the stability condition the training recipes are all implicitly chasing.

Contentless tokens

• Goyal, S., Ji, Z., Rawat, A. S., Menon, A. K., Kumar, S., and Nagarajan, V. (2024). “Think before you speak: Training Language Models With Pause Tokens.” ICLR. arXiv:2310.02226. Learnable pause tokens delay answer extraction and buy computation: +18% exact match on SQuAD at 1B. The precondition is strong: pauses must appear in both pretraining and fine-tuning. No code was released.
• Pfau, J., Merrill, W., and Bowman, S. R. (2024). Let's Think Dot by Dot: Hidden Computation in Transformer Language Models. arXiv:2404.15758. Transformers solve two tasks unsolvable in a single pass using nothing but repeated dots, with the admissible problems characterised by quantifier depth, but learning to exploit fillers needs dense, task-specific supervision and does not emerge from ordinary training.
• Brauer, K., Mayrink Verdun, C., and Marks, S. (2026). Reading Between the Dots: Decoding Hidden Computation across Filler Tokens. arXiv:2607.03502. Filler-token computation in frontier open-weight models defeats token-level monitoring outright, yet an unsupervised decoder on hidden states alone recovers the intermediate values at 80–95% accuracy, with causal validation by cache transplantation. Monitorability is a property of activations, not of surface tokens.

Negative results and critiques

This field produces rebuttals at an unusual rate, Several headline mechanisms did not survive replication.

• Vennam, S., Valente, D., Herel, D., and Kumaraguru, P. (2024). Rethinking Thinking Tokens: Understanding Why They Underperform in Practice. arXiv:2411.11371. Naive thinking tokens consistently underperform real chain of thought: a single repeated embedding produces noisy, inconsistent gradients.
• Yu, G. (2024). Do LLMs Really Think Step-by-step In Implicit Reasoning? arXiv:2411.15862. Models merely prompted to reason implicitly barely compute genuine intermediate steps and lean on memorised shortcuts; models specifically trained for it do show real internal computation. Both are brittle to reformatting.
• Wu, Y., Lu, K., Ren, X., Hu, J., Wu, X., et al. (2025). LLMs are Single-threaded Reasoners: Demystifying the Working Mechanism of Soft Thinking. arXiv:2508.03440. Direct rebuttal of the parallel-exploration claim: models given a soft mixture follow the single highest-probability component, so the method degenerates to greedy decoding unless noise is injected.
• Rizvi-Martel, M., Rabusseau, G., and Mosbach, M. (2026). The Illusion of Superposition? A Principled Analysis of Latent Thinking in Language Models. arXiv:2604.06374. Genuine superposition shows up only in models trained from scratch with continuous latent chain of thought. Under fine-tuning or training-free regimes it collapses to token commitment or shortcuts. Undercuts the family's central theoretical selling point.
• Sahoo, S., Chadha, A., Jain, V., and Chaudhary, D. (2026). When Shallow Wins: Silent Failures and the Depth-Accuracy Paradox in Latent Reasoning. arXiv:2603.03475. Reports 18.4% of correct predictions as stable and faithful against 81.6% “computationally inconsistent”, 8.8% silent failures, and $r = -0.21$ between reasoning quality and correctness. This is a contested diagnostic proposal. The reported correlation is hard to reconcile with the conditional means given alongside it, and the headline rates are threshold-based operational definitions rather than independent verification of reasoning errors: see families.
• Yang, R., Han, S., Zhang, Y., Wei, C., Shao, Y., et al. (2026). Stabilizing Recurrent Dynamics for Test-Time Scalable Latent Reasoning in Looped Language Models. arXiv:2605.26733. Looped performance peaks at some recurrence depth and then collapses with further looping; spectral-radius regularisation is proposed as the fix. Directly against the infinite-depth framing.

Faithfulness, monitorability and interpretability

• Turpin, M., Michael, J., Perez, E., and Bowman, S. R. (2023). “Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting.” NeurIPS. arXiv:2305.04388. Biasing cues shift answers by up to 36% across thirteen BIG-Bench Hard tasks while the stated reasoning systematically omits the bias.
• Lanham, T., Chen, A., Radhakrishnan, A., Steiner, B., Denison, C., et al. (2023). Measuring Faithfulness in Chain-of-Thought Reasoning. arXiv:2307.13702. Interventional metrics: truncation, inserted mistakes, paraphrase, filler substitution. Models often reach the same answer regardless of what the chain says, and faithfulness does not straightforwardly improve with scale.
• Roger, F., and Greenblatt, R. (2023). Preventing Language Models From Hiding Their Reasoning. arXiv:2310.18512. A fine-tuned model embeds reasoning steganographically in innocuous text — 80% opinion-matching against 54% without the hidden channel. Context-aware paraphrasing plus penalties for abnormal generations held estimated rates below about 3 bits per kilobyte across the schemes tested, on one instruction and one maths dataset. That estimate assumes a message is either transmitted perfectly or scrambled entirely, which the authors note underestimates capacity, so it bounds the tested encoders from below rather than covert capacity from above.
• Yang, S., Gribovskaya, E., Kassner, N., Geva, M., and Riedel, S. (2024). “Do Large Language Models Latently Perform Multi-Hop Reasoning?” ACL. arXiv:2402.16837. Evidence that models internally resolve a bridge entity and use it for the second hop, for certain relation types: reasoning was partly latent before anyone designed it that way. The follow-up arXiv:2411.16679 asks how much of this survives a shortcut analysis.
• Baker, B., Huizinga, J., Gao, L., Dou, Z., Guan, M. Y., et al. (2025). Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscation. arXiv:2503.11926. A weak monitor reading raw chains catches reward hacking during RL, but optimising the policy against that monitor yields obfuscated hacking that still passes it. The empirical basis for not training against chain-of-thought legibility.
• Lindsey, J., Gurnee, W., et al. (2025). On the Biology of a Large Language Model. Transformer Circuits. Attribution graphs show Claude 3.5 Haiku planning several tokens ahead, choosing rhyme words before writing the line that must end with them. Substantial reasoning precedes any emitted token.
• Chen, Y., Benton, J., Radhakrishnan, A., Uesato, J., Denison, C., et al. (2025). Reasoning Models Don't Always Say What They Think. arXiv:2505.05410. Across six hint categories, models verbalise using a hint they demonstrably used often below 20% of the time. Outcome-based RL improves faithfulness early and then plateaus without saturating.
• Lindsey, J. (2025). Emergent Introspective Awareness in Large Language Models. Transformer Circuits; later arXiv:2601.01828. Concept injection shows Claude Opus 4 and 4.1 sometimes detecting and naming a vector patched into the residual stream: roughly 20% of trials under favourable conditions. Failures of introspection remain the norm.
• Guan, M. Y., Wang, K., Carroll, M., Dou, Z., Wei, J., et al. (2025). Monitoring Monitorability. arXiv:2512.18311. Treats monitorability as a measurable and degradable property: thirteen evaluations across twenty-four environments in three archetypes.
• Dilgren, C., and Wiegreffe, S. (2026). Are Latent Reasoning Models Easily Interpretable? arXiv:2604.04902. The optimistic counterweight: latent reasoning tokens are frequently unnecessary, and where they carry the computation, traces decode 65–93% of the time, and substantially less often on incorrectly answered instances, which is where oversight matters most.
• Chang, S., Bai, T., Zhang, X., Ma, Q., Liu, Q., et al. (2026). Unlocking the Black Box of Latent Reasoning: An Interpretability-Guided Approach to Intervention. arXiv:2606.01243. Finds early latent vectors acting as causal hubs and builds training-free decode-time interventions on that basis.

Encoded reasoning

• Mathew, Y., Matthews, O., McCarthy, R., Velja, J., Schroeder de Witt, C., et al. (2024). Hidden in Plain Text: Emergence & Mitigation of Steganographic Collusion in LLMs. arXiv:2410.03768. Steganographic collusion emerging from ordinary mis-specified RL incentives rather than deliberate training, and robust to passive monitoring and paraphrase-based mitigation. The more troubling version of the result.
• Karpov, A., Adeleke, A., Cho, J., and Perez-Campanero, N. (2025). The Steganographic Potentials of Language Models. arXiv:2505.03439. RL-fine-tuned frontier models develop covert encoding schemes in realistic settings, though capability is rudimentary and decoding accuracy falls off with payload length.
• Zolkowski, A., Nishimura-Gasparian, K., McCarthy, R., Zimmermann, R., and Lindner, D. (2025). Early Signs of Steganographic Capabilities in Frontier LLMs. arXiv:2507.02737.

Scaling, padding and depth

• Schwethelm, K., Rueckert, D., and Kaissis, G. (2026). How Much Is One Recurrence Worth? Iso-Depth Scaling Laws for Looped Language Models. arXiv:2604.21106. Prices a recurrence in equivalent unique parameters, fitting $L = E + A(N_{\text{once}} + r^{\varphi} N_{\text{rec}})^{-\alpha} + BD^{-\beta}$ across $r \in \{1,2,4,8\}$ and roughly fifty-fold in training compute, with $\varphi \approx 0.46$ in the tested regime.
• Merrill, W., and Sabharwal, A. (2025). Exact Expressive Power of Transformers with Padding. arXiv:2505.18948. Padding as an alternative to chain of thought that avoids sequential decoding, with padding and looping analysed together under stated architecture assumptions.
• Merrill, W., and Sabharwal, A. (2025). A Little Depth Goes a Long Way: The Expressive Power of Log-Depth Transformers. arXiv:2503.03961. Constant depth is the wrong idealisation for short inputs: how far bounded depth suffices, and what growing it buys. The nearest thing the theory offers to task-specific depth guidance.
• Wierstra, D., Förster, A., Peters, J., and Schmidhuber, J. (2009). “Recurrent Policy Gradients.” PDF. Trains recurrent policies by return-weighted backpropagation. Cited here because it long predates, and contradicts, the claim that policy gradients cannot carry credit through a recurrence.

Probing and decoding latent computation

• Hagendorff, T., and Fabi, S. (2025). Beyond Chains of Thought: Benchmarking Latent-Space Reasoning Abilities in Large Language Models. arXiv:2504.10615. A 4,000-item benchmark that forces model-internal inference by requiring the answer to be signalled through the language of the first response token rather than through descriptive text, so the reasoning cannot be externalised. Eighteen models evaluated, best accuracy 74.7%. The authors state they cannot rule out heuristic exploitation under some conditions.
• Lu, W., Yang, Y., Lee, K., and Li, L. (2025). Latent Chain-of-Thought? Decoding the Depth-Recurrent Transformer. arXiv:2507.02199. Asks whether a depth-recurrent model's iterations correspond to anything like chain-of-thought steps, and attempts to decode them. The closest thing to interpretability tooling aimed specifically at recurrent depth.
• Nikolaou, G., Mencattini, T., Crisostomi, D., and Santilli, A. (2025). Language Models are Injective and Hence Invertible. arXiv:2510.15511. The input-to-state map is almost surely injective, at initialisation and after any finite number of gradient steps, so distinct prompts give distinct last-token representations despite non-injective activations and normalisation. Assumes real-analytic activations, which excludes ReLU. The recovery algorithm needs all per-position states at a layer; doing it from the final embedding alone is possible in principle and left open in practice. This concerns recovering the prompt from accessible activations: not recovering intermediate reasoning, and not recovering anything from the model's emitted text. Note also that the theorem is real-valued: a $d$-coordinate representation at $p$ bits per coordinate has at most $2^{dp}$ states, so injectivity over all length-$K$ prompts fails by counting once $|V|^K > 2^{dp}$.
• Yang, Z., Li, J., Xia, H., and Hu, X. (2025). Internal Chain-of-Thought: Empirical Evidence for Layer-wise Subtask Scheduling in LLMs. arXiv:2505.14530. Across fifteen two-step composite tasks, distinct subtasks are learned at different depths and executed in sequence across layers: multi-step structure inside a single forward pass.
• Du, Z., Wang, C., Bai, Y., and Cao, Y. (2025). Enabling Agents to Communicate Entirely in Latent Space. arXiv:2511.09149. Agent-to-agent communication that bypasses tokens on the argument that discretising internal states limits what can be transmitted. The concrete version of the neuralese concern in monitorability: multi-agent traffic with no readable channel by construction.

Further latent-reasoning methods

• Liu, H., Murty, S., Manning, C. D., and Csordás, R. (2025). Thoughtbubbles: an Unsupervised Method for Parallel Thinking in Latent Space. arXiv:2510.00219. A transformer variant that performs parallel latent computation natively and can be applied during pretraining, rather than requiring serially generated natural-language verbalisation.
• Koishekenov, Y., Lipani, A., and Cancedda, N. (2025). Encode, Think, Decode: Scaling test-time reasoning with recursive latent thoughts. arXiv:2510.07358. Motivated by interpretability work locating reasoning computation in a limited band of layers, recurses on that band rather than the whole stack.
• Zeng, Z., Li, Y., Song, K., and Wang, W. (2025). PonderLM-2: Pretraining LLM with Latent Thoughts in Continuous Space. arXiv:2509.23184. Moves the extra computational steps into pretraining, spending them on every token rather than at test time on a reasoning trace.
• Deng, S., Pang, W., Wei, Y., and Xu, L. (2025). LLM Latent Reasoning as Chain of Superposition. arXiv:2510.15522. Latent-SFT, addressing distributional misalignment and ambiguous chain definitions, on the premise that latent reasoning should function as a superposition of several reasoning paths. The replication evidence in families cuts against it.
• Asadulaev, A., Banerjee, S., Karray, F., and Takáč, M. (2025). Latent Reasoning in TRMs is Secretly a Policy Improvement Operator. arXiv:2511.16886. Questions the standard explanation that recursion simply buys effective depth, noting that recursively added layers trail one-pass models of the same feed-forward capacity.

Surveys and adjacent work

• Zhu, R.-J., Peng, T., Cheng, T., Qu, X., Huang, J., et al. (2025). A Survey on Latent Reasoning. arXiv:2507.06203. Thirty-three authors. Splits the field into vertical recurrence (activation-based, loop and hidden-state feedback) and horizontal recurrence (hidden-state-based, linear and gradient-state), and extends the framing to masked diffusion models.
• Chen, X., Zhao, A., Xia, H., Lu, X., Wang, H., et al. (2025). Reasoning Beyond Language: A Comprehensive Survey on Latent Chain-of-Thought Reasoning. arXiv:2505.16782. A token-wise horizontal versus layer-wise vertical taxonomy. Companion to the Awesome-Latent-CoT list.
• Li, J., Fu, Y., Fan, L., Liu, J., Shu, Y., et al. (2025). Implicit Reasoning in Large Language Models: A Comprehensive Survey. arXiv:2509.02350. Organises by mechanism of control, latent optimisation, signal-guided control, layer-recurrent execution, rather than by representation.
• Yu, X., Chen, Z., He, Y., Fu, T., Yang, J., et al. (2026). The Latent Space: Foundation, Evolution, Mechanism, Ability, and Outlook. arXiv:2604.02029. The broadest recent survey, treating latent reasoning as one instance of a wider latent-native paradigm. Whether that paradigm exists yet is itself contestable.
• Kuzina, A., Pióro, M., Whatmough, P., and Bejnordi, B. E. (2025). KaVa: Latent Reasoning via Compressed KV-Cache Distillation. arXiv:2510.02312. Distils a compressed teacher cache into a latent student, finding chain-of-thought caches highly redundant. The cache-as-scratchpad position.
• Kang, H., Zhang, W., Kuang, K., Majamaki, T., Jaitly, N., et al. (2025). LaDiR: Latent Diffusion Enhances LLMs for Text Reasoning. arXiv:2510.04573. Continuous latent representation combined with iterative diffusion refinement.
• Kang, H., Zhang, W., Kuang, K., Ma, X., and Qin, C. (2026). LaDi-RL: Latent Diffusion Reasoning Prevents Entropy Collapse in Reinforcement Learning. arXiv:2602.01705. Reinforcement learning over latent diffusion reasoning against verifiable rewards, including unit-test pass rates and exact answer correctness.
• Wang, T., and Reid, M. (2026). Tiny Recursive Reasoning with Mamba-2 Attention Hybrid. arXiv:2602.12078. Mamba-2 hybrid operators inside a latent recursive scaffold at roughly 7M parameters. Narrow in scale and task, but the operator question is being asked.
• Williams, M., and Türeci, H. E. (2026). Prioritize the Process, Not Just the Outcome: Rewarding Latent Thought Trajectories Improves Reasoning in Looped Language Models. arXiv:2602.10520. Reports that standard objectives such as GRPO have underperformed on looped models and that per-loop auxiliary credit helps. Note what this does and does not show: the paper's own account has the terminal signal backpropagating through the latent refinements, so this is an optimisation finding, not a demonstration that policy gradients cannot train recurrence. For a differentiable recurrence with gradients retained, a terminal reward's total derivative already includes every unrolled update.

Curated elsewhere: Awesome-Latent-Space and Awesome-Latent-CoT track new papers more frequently than this site does. Working on something that belongs here, or spotted an error? Open an issue on the repository.