570 citations · 2.6k across the 40 of their papers we have counts for
22 papers · 1 filter
Planning to Learn
Ian Osband
Policy-gradient methods are central to modern reinforcement learning, including LLM post-training. When they struggle, the usual suspects are exploration, credit assignment and act…
Delightful Exploration
Ian Osband
Most exploration algorithms search broadly until uncertainty is resolved. When the action space is too large to resolve within budget, practitioners default to -greedy…
Delightful Gradients Accelerate Corner Escape
Jincheng Mei, Ian Osband
Softmax policy gradient converges at , but its transient behavior near sub-optimal corners of the simplex can be exponentially slow. The bottleneck is self-trapping: negati…
Does This Gradient Spark Joy?
Ian Osband
Policy gradient computes a backward pass for every sample, even though the backward pass is expensive and most samples carry little learning value. The Delightful Policy Gradient (…
Delightful Policy Gradient
Ian Osband
Standard policy gradients weight each sampled action by advantage alone, regardless of how likely that action was under the current policy. This creates two pathologies: within a s…
Delightful Distributed Policy Gradient
Ian Osband
Distributed reinforcement learning trains on data from stale, buggy, or mismatched actors, producing actions with high surprisal (negative log-probability) under the learner's poli…
Approximate Thompson Sampling via Epistemic Neural Networks
Ian Osband, Zheng Wen, Seyed Mohammad Asghari +4
Thompson sampling (TS) is a popular heuristic for action selection, but it requires sampling from a posterior distribution. Unfortunately, this can become computationally intractab…
Robustness of Epinets against Distributional Shifts
Xiuyuan Lu, Ian Osband, Seyed Mohammad Asghari +4
Recent work introduced the epinet as a new approach to uncertainty modeling in deep learning. An epinet is a small neural network added to traditional neural networks, which, toget…
Ensembles for Uncertainty Estimation: Benefits of Prior Functions and Bootstrapping
Vikranth Dwaracherla, Zheng Wen, Ian Osband +3
In machine learning, an agent needs to estimate uncertainty to efficiently explore and adapt and to make effective decisions. A common approach to uncertainty estimation maintains…
The Neural Testbed: Evaluating Joint Predictions
Ian Osband, Zheng Wen, Seyed Mohammad Asghari +7
Predictive distributions quantify uncertainties ignored by point estimates. This paper introduces The Neural Testbed: an open-source benchmark for controlled and principled evaluat…
From Predictions to Decisions: The Importance of Joint Predictive Distributions
Zheng Wen, Ian Osband, Chao Qin +5
A fundamental challenge for any intelligent system is prediction: given some inputs, can you predict corresponding outcomes? Most work on supervised learning has focused on produci…
Epistemic Neural Networks
Ian Osband, Zheng Wen, Seyed Mohammad Asghari +4
Intelligence relies on an agent's knowledge of what it does not know. This capability can be assessed based on the quality of joint predictions of labels across multiple inputs. In…
Reinforcement Learning, Bit by Bit
Xiuyuan Lu, Benjamin Van Roy, Vikranth Dwaracherla +3
Reinforcement learning agents have demonstrated remarkable achievements in simulated environments. Data efficiency poses an impediment to carrying this success over to real environ…
Hypermodels for Exploration
Vikranth Dwaracherla, Xiuyuan Lu, Morteza Ibrahimi +3
We study the use of hypermodels to represent epistemic uncertainty and guide exploration. This generalizes and extends the use of ensembles to approximate Thompson sampling. The co…
Matrix games with bandit feedback
Brendan O'Donoghue, Tor Lattimore, Ian Osband
We study a version of the classical zero-sum matrix game with unknown payoff matrix and bandit feedback, where the players only observe each others actions and a noisy payoff. This…
Making Sense of Reinforcement Learning and Probabilistic Inference
Brendan O'Donoghue, Ian Osband, Catalin Ionescu
Reinforcement learning (RL) combines a control problem with statistical estimation: The system dynamics are not known to the agent, but can be learned through experience. A recent…
Behaviour Suite for Reinforcement Learning
Ian Osband, Yotam Doron, Matteo Hessel +11
This paper introduces the Behaviour Suite for Reinforcement Learning, or bsuite for short. bsuite is a collection of carefully-designed experiments that investigate core capabiliti…
Meta-learning of Sequential Strategies
Pedro A. Ortega, Jane X. Wang, Mark Rowland +21
In this report we review memory-based meta-learning as a tool for building sample-efficient strategies that learn from past experience to adapt to any task within a target class. O…
Scalable Coordinated Exploration in Concurrent Reinforcement Learning
Maria Dimakopoulou, Ian Osband, Benjamin Van Roy
We consider a team of reinforcement learning agents that concurrently operate in a common environment, and we develop an approach to efficient coordinated exploration that is suita…
A Tutorial on Thompson Sampling
Daniel Russo, Benjamin Van Roy, Abbas Kazerouni +2
Thompson sampling is an algorithm for online decision problems where actions are taken sequentially in a manner that must balance between exploiting what is known to maximize immed…
Noisy Networks for Exploration
Meire Fortunato, Mohammad Gheshlaghi Azar, Bilal Piot +9
We introduce NoisyNet, a deep reinforcement learning agent with parametric noise added to its weights, and show that the induced stochasticity of the agent's policy can be used to…
Deep Exploration via Bootstrapped DQN
Ian Osband, Charles Blundell, Alexander Pritzel +1
Efficient exploration in complex environments remains a major challenge for reinforcement learning. We propose bootstrapped DQN, a simple algorithm that explores in a computational…