35 citations · 243 across the 61 of their papers we have counts for
32 papers · 1 filter
The Weakest Link: Distilling LLM Reasoning with Worst-Case Constrained Reinforcement Learning
Matthieu Zimmer, Xiaotong Ji, Tu Nguyen +1
Distilling the reasoning capabilities of large language models (LLMs) into smaller students is a central challenge for efficient deployment. Current approaches face a fundamental t…
Does This Action Still Explain the Task? Reverse Scoring for Diffusion Language Model Agents
Jiacheng Qiu, Christopher E. Mower, Jan Peters +2
Diffusion-based large language models (dLLMs) promise to break the sequential latency bottleneck of autoregressive agents through parallel decoding, but recent evaluations show thi…
Tail-Influence Sampling for CVaR Policy Evaluation
Pauline Bourigault, Xiaotong Ji, Matthieu Zimmer +2
Policies with similar mean returns can differ sharply in rare failures, yet estimating lower-tail conditional value-at-risk (CVaR) accurately can require many costly rollouts. When…
Beam Search as Test-Time Self-Distillation via Counterfactual Contexts
Su Ee Tan, Xiaotong Ji, Rasul Tutunov +2
Self-Distillation Fine-Tuning (SDFT) enables a language model to act as its own teacher: by conditioning on a demonstration, the model produces an implicit reward via pointwise mut…
Composable Decoding on the Probability Simplex: Theory and Implementation
Xiaotong Ji, Ahmed Khaled Khamis, Rasul Tutunov +2
Decoding for large language models is typically treated as a collection of isolated sampling strategies, with limited theoretical understanding of the behaviours they induce and ho…
iSDFT: Information-Proximal Self-Distillation for Continual Learning in LLMs
Ahmed Khaled Khamis, Xiaotong Ji, Hassan Jaber +4
On-policy self-distillation fine-tuning (SDFT) learns new skills from demonstrations while reducing forgetting, but it always distils toward the full demonstration-conditioned teac…
Trust Region Inverse Reinforcement Learning: Explicit Dual Ascent using Local Policy Updates
Anish Diwan, Davide Tateo, Christopher E. Mower +3
Inverse reinforcement learning (IRL) is typically formulated as maximizing entropy subject to matching the distribution of expert trajectories. Classical (dual-ascent) IRL guarante…
Efficient and Principled Scientific Discovery through Bayesian Optimization: A Tutorial
Zhongwei Yu, Rasul Tutunov, Alexandre Max Maraval +13
Traditional scientific discovery relies on an iterative hypothesise-experiment-refine cycle that has driven progress for centuries, but its intuitive, ad-hoc implementation often w…
The -Combinator for LLMs: Solving Long-Context Rot with -Calculus
Amartya Roy, Rasul Tutunov, Xiaotong Ji +2
LLMs are increasingly used as general-purpose reasoners, but long inputs remain bottlenecked by a fixed context window. Recursive Language Models (RLMs) address this by externalisi…
Decoding as Optimisation on the Probability Simplex: From Top-K to Top-P (Nucleus) to Best-of-K Samplers
Xiaotong Ji, Rasul Tutunov, Matthieu Zimmer +1
Decoding sits between a language model and everything we do with it, yet it is still treated as a heuristic knob-tuning exercise. We argue decoding should be understood as a princi…
Scalable Power Sampling: Unlocking Efficient, Training-Free Reasoning for LLMs via Distribution Sharpening
Xiaotong Ji, Rasul Tutunov, Matthieu Zimmer +1
Reinforcement learning (RL) post-training is a dominant approach for improving the reasoning performance of large language models (LLMs), yet growing evidence suggests that its gai…
SuRe: Surprise-Driven Prioritised Replay for Continual LLM Learning
Hugo Hazard, Zafeirios Fountas, Martin A. Benfeghoul +3
Continual learning, one's ability to adapt to a sequence of tasks without forgetting previously acquired knowledge, remains a major challenge in machine learning and a key gap betw…
Bottlenecked Transformers: Periodic KV Cache Consolidation for Generalised Reasoning
Adnan Oomerjee, Zafeirios Fountas, Haitham Bou-Ammar +1
Transformer LLMs have been shown to exhibit strong reasoning ability that scales with inference-time compute, most prominently through token-space "thinking" chains of thought. A g…
Al-Khwarizmi: Discovering Physical Laws with Foundation Models
Christopher E. Mower, Haitham Bou-Ammar
Inferring physical laws from data is a central challenge in science and engineering, including but not limited to healthcare, physical sciences, biosciences, social sciences, susta…
Many of Your DPOs are Secretly One: Attempting Unification Through Mutual Information
Rasul Tutnov, Antoine Grosnit, Haitham Bou-Ammar
Post-alignment of large language models (LLMs) is critical in improving their utility, safety, and alignment with human intentions. Direct preference optimisation (DPO) has become…
Kolb-Based Experiential Learning for Generalist Agents with Human-Level Kaggle Data Science Performance
Antoine Grosnit, Alexandre Maraval, Refinath S N +16
Human expertise emerges through iterative cycles of interaction, reflection, and internal model updating, which are central to cognitive theories such as Kolb's experiential learni…
ShortCircuit: AlphaZero-Driven Circuit Design
Dimitrios Tsaras, Antoine Grosnit, Lei Chen +3
Chip design relies heavily on generating Boolean circuits, such as AND-Inverter Graphs (AIGs), from functional descriptions like truth tables. This generation operation is a key pr…
Bayesian Reward Models for LLM Alignment
Adam X. Yang, Maxime Robeyns, Thomas Coste +4
To ensure that large language model (LLM) responses are helpful and non-toxic, a reward model trained on human preference data is usually used. LLM responses with high rewards are…
Framework and Benchmarks for Combinatorial and Mixed-variable Bayesian Optimization
Kamil Dreczkowski, Antoine Grosnit, Haitham Bou Ammar
This paper introduces a modular framework for Mixed-variable and Combinatorial Bayesian Optimization (MCBO) to address the lack of systematic benchmarking and standardized evaluati…
Are Random Decompositions all we need in High Dimensional Bayesian Optimisation?
Juliusz Ziomek, Haitham Bou-Ammar
Learning decompositions of expensive-to-evaluate black-box functions promises to scale Bayesian optimisation (BO) to high-dimensional problems. However, the success of these techni…
Contextual Causal Bayesian Optimisation
Vahan Arsenyan, Antoine Grosnit, Haitham Bou-Ammar +1
We introduce a unified framework for contextual and causal Bayesian optimisation, which aims to design intervention policies maximising the expectation of a target variable. Our ap…
Reinforcement Learning in Presence of Discrete Markovian Context Evolution
Hang Ren, Aivar Sootla, Taher Jafferjee +3
We consider a context-dependent Reinforcement Learning (RL) setting, which is characterized by: a) an unknown finite number of not directly observable contexts; b) abrupt (disconti…
Saute RL: Almost Surely Safe Reinforcement Learning Using State Augmentation
Aivar Sootla, Alexander I. Cowen-Rivers, Taher Jafferjee +4
Satisfying safety constraints almost surely (or with probability one) can be critical for the deployment of Reinforcement Learning (RL) in real-life applications. For example, plan…
High-Dimensional Bayesian Optimisation with Variational Autoencoders and Deep Metric Learning
Antoine Grosnit, Rasul Tutunov, Alexandre Max Maraval +9
We introduce a method combining variational autoencoders (VAEs) and deep metric learning to perform Bayesian optimisation (BO) over high-dimensional and structured input spaces. By…