21 citations · 96 across the 24 of their papers we have counts for
31 papers
Tail-Influence Sampling for CVaR Policy Evaluation
Pauline Bourigault, Xiaotong Ji, Matthieu Zimmer +2
Policies with similar mean returns can differ sharply in rare failures, yet estimating lower-tail conditional value-at-risk (CVaR) accurately can require many costly rollouts. When…
Beam Search as Test-Time Self-Distillation via Counterfactual Contexts
Su Ee Tan, Xiaotong Ji, Rasul Tutunov +2
Self-Distillation Fine-Tuning (SDFT) enables a language model to act as its own teacher: by conditioning on a demonstration, the model produces an implicit reward via pointwise mut…
Composable Decoding on the Probability Simplex: Theory and Implementation
Xiaotong Ji, Ahmed Khaled Khamis, Rasul Tutunov +2
Decoding for large language models is typically treated as a collection of isolated sampling strategies, with limited theoretical understanding of the behaviours they induce and ho…
iSDFT: Information-Proximal Self-Distillation for Continual Learning in LLMs
Ahmed Khaled Khamis, Xiaotong Ji, Hassan Jaber +4
On-policy self-distillation fine-tuning (SDFT) learns new skills from demonstrations while reducing forgetting, but it always distils toward the full demonstration-conditioned teac…
The Model Knows, the Decoder Finds: Future Value Guided Particle Power Sampling
Tu Nguyen, Matthieu Zimmer, Rasul Tutunov +2
A recurring pattern in "reasoning without training" is that base LLMs already assign non-trivial probability mass to correct multi-step solutions; the bottleneck is locating these…
Risk-Controlled Lean-as-Judge for Natural-Language Mathematical Reasoning
Pauline Bourigault, Xiaotong Ji, Matthieu Zimmer +2
Lean is increasingly used to judge natural-language mathematical answers, but its signal is partial: many answers never formalize, and a failed proof may reflect an ill-typed state…
Efficient and Principled Scientific Discovery through Bayesian Optimization: A Tutorial
Zhongwei Yu, Rasul Tutunov, Alexandre Max Maraval +13
Traditional scientific discovery relies on an iterative hypothesise-experiment-refine cycle that has driven progress for centuries, but its intuitive, ad-hoc implementation often w…
The -Combinator for LLMs: Solving Long-Context Rot with -Calculus
Amartya Roy, Rasul Tutunov, Xiaotong Ji +2
LLMs are increasingly used as general-purpose reasoners, but long inputs remain bottlenecked by a fixed context window. Recursive Language Models (RLMs) address this by externalisi…
Decoding as Optimisation on the Probability Simplex: From Top-K to Top-P (Nucleus) to Best-of-K Samplers
Xiaotong Ji, Rasul Tutunov, Matthieu Zimmer +1
Decoding sits between a language model and everything we do with it, yet it is still treated as a heuristic knob-tuning exercise. We argue decoding should be understood as a princi…
Scalable Power Sampling: Unlocking Efficient, Training-Free Reasoning for LLMs via Distribution Sharpening
Xiaotong Ji, Rasul Tutunov, Matthieu Zimmer +1
Reinforcement learning (RL) post-training is a dominant approach for improving the reasoning performance of large language models (LLMs), yet growing evidence suggests that its gai…
Model-Based and Sample-Efficient AI-Assisted Math Discovery in Sphere Packing
Rasul Tutunov, Alexandre Maraval, Antoine Grosnit +3
Sphere packing, Hilbert's eighteenth problem, asks for the densest arrangement of congruent spheres in n-dimensional Euclidean space. Although relevant to areas such as cryptograph…
Bourbaki: Self-Generated and Goal-Conditioned MDPs for Theorem Proving
Matthieu Zimmer, Xiaotong Ji, Rasul Tutunov +3
Reasoning remains a challenging task for large language models (LLMs), especially within the logically constrained environment of automated theorem proving (ATP), due to sparse rew…
Why Can Large Language Models Generate Correct Chain-of-Thoughts?
Rasul Tutunov, Antoine Grosnit, Juliusz Ziomek +2
This paper delves into the capabilities of large language models (LLMs), specifically focusing on advancing the theoretical comprehension of chain-of-thought prompting. We investig…
Sample-Efficient Optimisation with Probabilistic Transformer Surrogates
Alexandre Maraval, Matthieu Zimmer, Antoine Grosnit +3
Faced with problems of increasing complexity, recent research in Bayesian Optimisation (BO) has focused on adapting deep probabilistic models as flexible alternatives to Gaussian P…
Self-consistent Gradient-like Eigen Decomposition in Solving Schrödinger Equations
Xihan Li, Xiang Chen, Rasul Tutunov +3
The Schrödinger equation is at the heart of modern quantum mechanics. Since exact solutions of the ground state are typically intractable, standard approaches approximate Schröding…
AntBO: Towards Real-World Automated Antibody Design with Combinatorial Bayesian Optimisation
Asif Khan, Alexander I. Cowen-Rivers, Antoine Grosnit +12
Antibodies are canonically Y-shaped multimeric proteins capable of highly specific molecular recognition. The CDRH3 region located at the tip of variable chains of an antibody domi…
BOiLS: Bayesian Optimisation for Logic Synthesis
Antoine Grosnit, Cedric Malherbe, Rasul Tutunov +3
Optimising the quality-of-results (QoR) of circuits during logic synthesis is a formidable challenge necessitating the exploration of exponentially sized search spaces. While exper…
High-Dimensional Bayesian Optimisation with Variational Autoencoders and Deep Metric Learning
Antoine Grosnit, Rasul Tutunov, Alexandre Max Maraval +9
We introduce a method combining variational autoencoders (VAEs) and deep metric learning to perform Bayesian optimisation (BO) over high-dimensional and structured input spaces. By…
Efficient Semi-Implicit Variational Inference
Vincent Moens, Hang Ren, Alexandre Maraval +3
In this paper, we propose CI-VI an efficient and scalable solver for semi-implicit variational inference (SIVI). Our method, first, maps SIVI's evidence lower bound (ELBO) to a for…
Are we Forgetting about Compositional Optimisers in Bayesian Optimisation?
Antoine Grosnit, Alexander I. Cowen-Rivers, Rasul Tutunov +3
Bayesian optimisation presents a sample-efficient methodology for global optimisation. Within this framework, a crucial performance-determining subroutine is the maximisation of th…
HEBO Pushing The Limits of Sample-Efficient Hyperparameter Optimisation
Alexander I. Cowen-Rivers, Wenlong Lyu, Rasul Tutunov +8
In this work we rigorously analyse assumptions inherent to black-box optimisation hyper-parameter tuning tasks. Our results on the Bayesmark benchmark indicate that heteroscedastic…
Compositional ADAM: An Adaptive Compositional Solver
Rasul Tutunov, Minne Li, Alexander I. Cowen-Rivers +2
In this paper, we present C-ADAM, the first adaptive solver for compositional problems involving a non-linear functional nesting of expected values. We proof that C-ADAM converges…
Derivative-Free & Order-Robust Optimisation
Victor Gabillon, Rasul Tutunov, Michal Valko +1
In this paper, we formalise order-robust optimisation as an instance of online learning minimising simple regret, and propose Vroom, a zero'th order optimisation algorithm capable…
-Rank: Practically Scaling -Rank through Stochastic Optimisation
Yaodong Yang, Rasul Tutunov, Phu Sakulwongtana +1
Recently, -Rank, a graph-based algorithm, has been proposed as a solution to ranking joint policy profiles in large scale multi-agent systems. -Rank claimed tractability thro…