Publications (9)
Reinforcing General Reasoning without Verifiers
Xiangxin Zhou, Zichen Liu, Anya Sims +6
The recent paradigm shift towards training large language models (LLMs) using DeepSeek-R1-Zero-style reinforcement learning (RL) on verifiable rewards has led to impressive advance…
Deep Thinking by Markov Chain of Continuous Thoughts
Jiayu Liu, Zhenya Huang, Xuan Yang +6
Transformer-based models can perform complicated reasoning by generating reasoning paths token by token. While effective, this approach often requires generating thousands of token…
The Edge-of-Reach Problem in Offline Model-Based Reinforcement Learning
Anya Sims, Cong Lu, Jakob Foerster +1
Offline reinforcement learning aims to train agents from pre-collected datasets. However, this comes with the added challenge of estimating the value of behaviors not covered in th…
Evolution Strategies at the Hyperscale
Bidipta Sarkar, Mattie Fellows, Juan Agustin Duque +17
Evolution Strategies (ES) is a class of powerful black-box optimisation methods that are highly parallelisable and can handle non-differentiable and noisy objectives. However, naï…
Stochasticity in Tokenisation Improves Robustness
Sophie Steger, Rui Li, Sofiane Ennadir +4
The widespread adoption of large language models (LLMs) has increased concerns about their robustness. Vulnerabilities in perturbations of tokenisation of the input indicate that m…
StochasTok: Improving Fine-Grained Subword Understanding in LLMs
Anya Sims, Thom Foster, Klara Kaleb +5
Subword-level understanding is integral to numerous tasks, including understanding multi-digit numbers, spelling mistakes, abbreviations, rhyming, and wordplay. Despite this, curre…
Learning to Reason at the Frontier of Learnability
Thomas Foster, Anya Sims, Johannes Forkel +2
Reinforcement learning is now widely adopted as the final stage of large language model training, especially for reasoning-style tasks such as maths problems. Typically, models att…
CODA: Coordination via On-Policy Diffusion for Multi-Agent Offline Reinforcement Learning
Marcel Hedman, Kale-ab Abebe Tessera, Juan Claude Formanek +5
Offline multi-agent reinforcement learning (MARL) enables policy learning from fixed datasets, but is prone to coordination failure: agents trained on static, off-policy data conve…
GEM: A Gym for Agentic LLMs
Zichen Liu, Anya Sims, Keyu Duan +16
The training paradigm for large language models (LLMs) is moving from static datasets to experience-based learning, where agents acquire skills via interacting with complex environ…