58 citations · 239 across the 22 of their papers we have counts for
25 papers
Measuring Uncertainty Calibration
Kamil Ciosek, Nicolò Felicioni, Sina Ghiassian +6
We make two contributions to the problem of estimating the calibration error of a binary classifier from a finite dataset. First, we provide an upper bound for any classifier…
Gradient Prediction with Control Variates in the Cheap-Forward Regime
Kamil Ciosek, Nicolò Felicioni, Juan Elenter +1
We study whether otherwise-idle inference resources could reduce the scarce-GPU cost of training. Our analysis uses a simulated compute ledger in which fleet work is billed at a fr…
Hallucination Detection on a Budget: Efficient Bayesian Estimation of Semantic Entropy
Kamil Ciosek, Nicolò Felicioni, Sina Ghiassian
Detecting whether an LLM hallucinates is an important research challenge. One promising way of doing so is to estimate the semantic entropy (Farquhar et al., 2024) of the distribut…
On the Importance of Uncertainty in Decision-Making with Large Language Models
Nicolò Felicioni, Lucas Maystre, Sina Ghiassian +1
We investigate the role of uncertainty in decision-making problems with natural language as input. For such tasks, using Large Language Models as agents has become the norm. Howeve…
Automatic Music Playlist Generation via Simulation-based Reinforcement Learning
Federico Tomasi, Joseph Cauteruccio, Surya Kanoria +3
Personalization of playlists is a common feature in music streaming services, but conventional techniques, such as collaborative filtering, rely on explicit assumptions regarding c…
Impatient Bandits: Optimizing Recommendations for the Long-Term Without Delay
Thomas M. McDonald, Lucas Maystre, Mounia Lalmas +2
Recommender systems are a ubiquitous feature of online platforms. Increasingly, they are explicitly tasked with increasing users' long-term satisfaction. In this context, we study…
A Strong Baseline for Batch Imitation Learning
Matthew Smith, Lucas Maystre, Zhenwen Dai +1
Imitation of expert behaviour is a highly desirable and safe approach to the problem of sequential decision making. We provide an easy-to-implement, novel algorithm for imitation l…
Imitation Learning by Reinforcement Learning
Kamil Ciosek
Imitation learning algorithms learn a policy from demonstrations of expert behavior. We show that, for deterministic experts, imitation learning can be done by reduction to reinfor…
Information Directed Reward Learning for Reinforcement Learning
David Lindner, Matteo Turchetta, Sebastian Tschiatschek +2
For many reinforcement learning (RL) applications, specifying a reward is difficult. This paper considers an RL setting where the agent obtains information about the reward only by…
Estimating -Rank by Maximizing Information Gain
Tabish Rashid, Cheng Zhang, Kamil Ciosek
Game theory has been increasingly applied in settings where the game is not known outright, but has to be estimated by sampling. For example, meta-games that arise in multi-agent e…
Regularized Policies are Reward Robust
Hisham Husain, Kamil Ciosek, Ryota Tomioka
Entropic regularization of policies in Reinforcement Learning (RL) is a commonly used heuristic to ensure that the learned policy explores the state-space sufficiently before overf…
Evaluating the Robustness of Collaborative Agents
Paul Knott, Micah Carroll, Sam Devlin +4
In order for agents trained by deep reinforcement learning to work alongside humans in realistic settings, we will need to ensure that the agents are \emph{robust}. Since the real…
Deep Interactive Bayesian Reinforcement Learning via Meta-Learning
Luisa Zintgraf, Sam Devlin, Kamil Ciosek +2
Agents that interact with other agents often do not know a priori what the other agents' strategies are, but have to maximise their own online return while interacting with and lea…
DRIFT: Deep Reinforcement Learning for Functional Software Testing
Luke Harries, Rebekah Storan Clarke, Timothy Chapman +10
Efficient software testing is essential for productive software development and reliable user experiences. As human testing is inefficient and expensive, automated software testing…
Discount Factor as a Regularizer in Reinforcement Learning
Ron Amit, Ron Meir, Kamil Ciosek
Specifying a Reinforcement Learning (RL) task involves choosing a suitable planning horizon, which is typically modeled by a discount factor. It is known that applying RL algorithm…
Generalization in Reinforcement Learning with Selective Noise Injection and Information Bottleneck
Maximilian Igl, Kamil Ciosek, Yingzhen Li +4
The ability for policies to generalize to new environments is key to the broad application of RL agents. A promising approach to prevent an agent's policy from overfitting to a lim…
Better Exploration with Optimistic Actor-Critic
Kamil Ciosek, Quan Vuong, Robert Loftin +1
Actor-critic methods, a type of model-free Reinforcement Learning, have been successfully applied to challenging tasks in continuous control, often achieving state-of-the art perfo…
Multi-task Batch Reinforcement Learning with Metric Learning
Jiachen Li, Quan Vuong, Shuang Liu +5
We tackle the Multi-task Batch Reinforcement Learning problem. Given multiple datasets collected from different tasks, we train a multi-task policy to perform well in unseen tasks…
Fourier Policy Gradients
Matthew Fellows, Kamil Ciosek, Shimon Whiteson
We propose a new way of deriving policy gradient updates for reinforcement learning. Our technique, based on Fourier analysis, recasts integrals that arise with expected policy gra…
Expected Policy Gradients for Reinforcement Learning
Kamil Ciosek, Shimon Whiteson
We propose expected policy gradients (EPG), which unify stochastic policy gradients (SPG) and deterministic policy gradients (DPG) for reinforcement learning. Inspired by expected…
Expected Policy Gradients
Kamil Ciosek, Shimon Whiteson
We propose expected policy gradients (EPG), which unify stochastic policy gradients (SPG) and deterministic policy gradients (DPG) for reinforcement learning. Inspired by expected…
Alternating Optimisation and Quadrature for Robust Control
Supratik Paul, Konstantinos Chatzilygeroudis, Kamil Ciosek +3
Bayesian optimisation has been successfully applied to a variety of reinforcement learning problems. However, the traditional approach for learning optimal policies in simulators d…
Value Iteration with Options and State Aggregation
Kamil Ciosek, David Silver
This paper presents a way of solving Markov Decision Processes that combines state abstraction and temporal abstraction. Specifically, we combine state aggregation with the options…
Properties of the Least Squares Temporal Difference learning algorithm
Kamil Ciosek
This paper presents four different ways of looking at the well-known Least Squares Temporal Differences (LSTD) algorithm for computing the value function of a Markov Reward Process…