27 citations · 149 across the 34 of their papers we have counts for
35 papers
Active, anytime-valid risk controlling prediction sets
Ziyu Xu, Nikos Karampatziakis, Paul Mineiro
Rigorously establishing the safety of black-box machine learning models concerning critical risk measures is important for providing guarantees about model behavior. Recently, Bate…
Efficient Contextual Bandits with Uninformed Feedback Graphs
Mengxiao Zhang, Yuheng Zhang, Haipeng Luo +1
Bandits with feedback graphs are powerful online learning models that interpolate between the full information and classic bandit problems, capturing many real-life applications. A…
Time-uniform confidence bands for the CDF under nonstationarity
Paul Mineiro, Steven R. Howard
Estimation of the complete distribution of a random variable is a useful primitive for both manual and automated decision making. This problem has received extensive attention in t…
Practical Contextual Bandits with Feedback Graphs
Mengxiao Zhang, Yuheng Zhang, Olga Vrousgou +2
While contextual bandit has a mature theory, effectively leveraging different feedback patterns to enhance the pace of learning remains unclear. Bandits with feedback graphs, which…
Infinite Action Contextual Bandits with Reusable Data Exhaust
Mark Rucker, Yinglun Zhu, Paul Mineiro
For infinite action contextual bandits, smoothed regret and reduction to regression results in state-of-the-art online performance with computational cost independent of the action…
Towards Data-Driven Offline Simulations for Online Reinforcement Learning
Shengpu Tang, Felipe Vieira Frujeri, Dipendra Misra +4
Modern decision-making systems, from robots to web recommendation engines, are expected to adapt: to user preferences, changing circumstances or even new tasks. Yet, it is still un…
Personalized Reward Learning with Interaction-Grounded Learning (IGL)
Jessica Maghakian, Paul Mineiro, Kishan Panaganti +3
In an era of countless content offerings, recommender systems alleviate information overload by providing users with personalized content suggestions. Due to the scarcity of explic…
Deploying a Steered Query Optimizer in Production at Microsoft
Wangda Zhang, Matteo Interlandi, Paul Mineiro +6
Modern analytical workloads are highly heterogeneous and massively complex, making generic query optimizers untenable for many customers and scenarios. As a result, it is important…
A lower confidence sequence for the changing mean of non-negative right heavy-tailed observations with bounded mean
Paul Mineiro
A confidence sequence (CS) is an anytime-valid sequential inference primitive which produces an adapted sequence of sets for a predictable parameter sequence with a time-uniform co…
Conditionally Risk-Averse Contextual Bandits
Mónika Farsang, Paul Mineiro, Wangda Zhang
Contextual bandits with average-case statistical guarantees are inadequate in risk-averse situations because they might trade off degraded worst-case behaviour for better average p…
Anytime-valid off-policy inference for contextual bandits
Ian Waudby-Smith, Lili Wu, Aaditya Ramdas +2
Contextual bandit algorithms are ubiquitous tools for active sequential experimentation in healthcare and the tech industry. They involve online learning algorithms that adaptively…
Contextual Bandits with Smooth Regret: Efficient Learning in Continuous Action Spaces
Yinglun Zhu, Paul Mineiro
Designing efficient general-purpose contextual bandit algorithms that work with large -- or even continuous -- action spaces would facilitate application to important scenarios suc…
Contextual Bandits with Large Action Spaces: Made Practical
Yinglun Zhu, Dylan J. Foster, John Langford +1
A central problem in sequential decision making is to develop algorithms that are practical and computationally efficient, yet support the use of flexible, general-purpose models.…
Interaction-Grounded Learning with Action-inclusive Feedback
Tengyang Xie, Akanksha Saran, Dylan J. Foster +5
Consider the problem setting of Interaction-Grounded Learning (IGL), in which a learner's goal is to optimally interact with the environment with no explicit reward to ground its p…
Interaction-Grounded Learning
Tengyang Xie, John Langford, Paul Mineiro +1
Consider a prosthetic arm, learning to adapt to its user's control signals. We propose Interaction-Grounded Learning for this novel setting, in which a learner's goal is to interac…
ChaCha for Online AutoML
Qingyun Wu, Chi Wang, John Langford +2
We propose the ChaCha (Champion-Challengers) algorithm for making an online choice of hyperparameters in online learning settings. ChaCha handles the process of determining a champ…
Bellman-consistent Pessimism for Offline Reinforcement Learning
Tengyang Xie, Ching-An Cheng, Nan Jiang +2
The use of pessimism, when reasoning about datasets lacking exhaustive exploration has recently gained prominence in offline reinforcement learning. Despite the robustness it adds…
Improving Long-Term Metrics in Recommendation Systems using Short-Horizon Reinforcement Learning
Bogdan Mazoure, Paul Mineiro, Pavithra Srinath +3
We study session-based recommendation scenarios where we want to recommend items to users during sequential interactions to improve their long-term utility. Optimizing a long-term…
Off-policy Confidence Sequences
Nikos Karampatziakis, Paul Mineiro, Aaditya Ramdas
We develop confidence bounds that hold uniformly over time for off-policy evaluation in the contextual bandit setting. These confidence sequences are based on recent ideas from mar…
Lessons from Contextual Bandit Learning in a Customer Support Bot
Nikos Karampatziakis, Sebastian Kochman, Jade Huang +3
In this work, we describe practical lessons we have learned from successfully using contextual bandits (CBs) to improve key business metrics of the Microsoft Virtual Agent for cust…
Empirical Likelihood for Contextual Bandits
Nikos Karampatziakis, John Langford, Paul Mineiro
We propose an estimator and confidence interval for computing the value of a policy from off-policy data in the contextual bandit setting. To this end we apply empirical likelihood…
Contextual Memory Trees
Wen Sun, Alina Beygelzimer, Hal Daumé +2
We design and study a Contextual Memory Tree (CMT), a learning memory controller that inserts new memories into an experience store of unbounded size. It is designed to efficiently…
Logarithmic Time One-Against-Some
Hal Daume, Nikos Karampatziakis, John Langford +1
We create a new online reduction of multiclass classification to binary classification for which training and prediction time scale logarithmically with the number of classes. Comp…
Active Information Acquisition
He He, Paul Mineiro, Nikos Karampatziakis
We propose a general framework for sequential and dynamic acquisition of useful information in order to solve a particular task. While our goal could in principle be tackled by gen…