Publications (47)
Development and validation of an interpretable machine learning-based calculator for predicting 5-year weight trajectories after bariatric surgery: a multinational retrospective cohort SOPHIA study
Patrick Saux, Pierre Bauvin, Violeta Raverdy +31
Background Weight loss trajectories after bariatric surgery vary widely between individuals, and predicting weight loss before the operation remains challenging. We aimed to develo…
Evaluating Interpretable Reinforcement Learning by Distilling Policies into Programs
Hector Kohler, Quentin Delfosse, Waris Radji +2
There exist applications of reinforcement learning like medicine where policies need to be ''interpretable'' by humans. User studies have shown that some policy classes might be mo…
Fast Reinforcement Learning with Large Action Sets using Error-Correcting Output Codes for MDP Factorization
Gabriel Dulac-Arnold, Ludovic Denoyer, Philippe Preux +1
The use of Reinforcement Learning in real-world scenarios is strongly limited by issues of scale. Most RL learning algorithms are unable to deal with problems composed of hundreds…
Reinforcement-learning robotic sailboats: simulator and preliminary results
Eduardo Charles Vasconcellos, Ronald M Sampaio, André P D Araújo +7
This work focuses on the main challenges and problems in developing a virtual oceanic environment reproducing real experiments using Unmanned Surface Vehicles (USV) digital twins.…
Low-Rank Projections of GCNs Laplacian
Nathan Grinsztajn, Philippe Preux, Edouard Oyallon
In this work, we study the behavior of standard models for community detection under spectral manipulations. Through various ablation experiments, we evaluate the impact of bandpas…
PAQA: Toward ProActive Open-Retrieval Question Answering
Pierre Erbacher, Jian-Yun Nie, Philippe Preux +1
Conversational systems have made significant progress in generating natural language responses. However, their potential as conversational search systems is currently limited due t…
Multiple functional regression with both discrete and continuous covariates
Hachem Kadri, Philippe Preux, Emmanuel Duflos +1
In this paper we present a nonparametric method for extending functional regression methodology to the situation where more than one functional covariate is used to predict a funct…
Entropy Regularized Reinforcement Learning with Cascading Networks
Riccardo Della Vecchia, Alena Shilova, Philippe Preux +1
Deep Reinforcement Learning (Deep RL) has had incredible achievements on high dimensional problems, yet its learning process remains unstable even on the simplest tasks. Deep RL us…
Leveraging Structural Constraints for Diffusion-based Neural TSP Solvers
Mickaël Basson, Philippe Preux
Neural combinatorial optimization has recently achieved strong results on the Euclidean Traveling Salesman Problem (TSP) using generative models such as diffusion and consistency m…
When (and How) to Trust the Expert: Diagnosing Query-Time Expert-Guided Reinforcement Learning
Yann Berthelot, Philippe Preux, Riad Akrour
Many continuous-control problems ship with a competent but suboptimal controller (a tuned PID, a hand-designed gait). A growing family of methods uses such controllers as queryable…
Augmenting Ad-Hoc IR Dataset for Interactive Conversational Search
Pierre Erbacher, Jian-Yun Nie, Philippe Preux +1
A peculiarity of conversational search systems is that they involve mixed-initiatives such as system-generated query clarifying questions. Evaluating those systems at a large scale…
Visual Reasoning with Multi-hop Feature Modulation
Florian Strub, Mathieu Seurin, Ethan Perez +5
Recent breakthroughs in computer vision and natural language processing have spurred interest in challenging multi-modal tasks such as visual question-answering and visual dialogue…
A Unified View of TD Algorithms; Introducing Full-Gradient TD and Equi-Gradient Descent TD
Manuel Loth, Philippe Preux
This paper addresses the issue of policy evaluation in Markov Decision Processes, using linear function approximation. It provides a unified view of algorithms such as TD(lambda),…
A Generative Model of Software Dependency Graphs to Better Understand Software Evolution
Vincenzo Musco, Martin Monperrus, Philippe Preux
Software systems are composed of many interacting elements. A natural way to abstract over software systems is to model them as graphs. In this paper we consider software dependenc…
Correctness Attraction: A Study of Stability of Software Behavior Under Runtime Perturbation
Benjamin Danglot, Philippe Preux, Benoit Baudry +1
Can the execution of a software be perturbed without breaking the correctness of the output? In this paper, we devise a novel protocol to answer this rarely investigated question.…
Cold-start Problems in Recommendation Systems via Contextual-bandit Algorithms
Hai Thanh Nguyen, Jérémie Mary, Philippe Preux
In this paper, we study a cold-start problem in recommendation systems where we have completely new users entered the systems. There is not any interaction or feedback of the new u…
Improving offline evaluation of contextual bandit algorithms via bootstrapping techniques
Olivier Nicol, Jérémie Mary, Philippe Preux
In many recommendation applications such as news recommendation, the items that can be rec- ommended come and go at a very fast pace. This is a challenge for recommender systems (R…
I'm sorry Dave, I'm afraid I can't do that, Deep Q-learning from forbidden action
Mathieu Seurin, Philippe Preux, Olivier Pietquin
The use of Reinforcement Learning (RL) is still restricted to simulation or to enhance human-operated systems through recommendations. Real-world environments (e.g. industrial robo…
Recurrent Neural Networks for Long and Short-Term Sequential Recommendation
Kiewan Villatel, Elena Smirnova, Jérémie Mary +1
Recommender systems objectives can be broadly characterized as modeling user preferences over short-or long-term time horizon. A large body of previous research studied long-term r…
Operator-valued Kernels for Learning from Functional Response Data
Hachem Kadri, Emmanuel Duflos, Philippe Preux +3
In this paper we consider the problems of supervised classification and regression in the case where attributes and labels are functions: a data is represented by a set of function…
Geometric Deep Reinforcement Learning for Dynamic DAG Scheduling
Nathan Grinsztajn, Olivier Beaumont, Emmanuel Jeannot +1
In practice, it is quite common to face combinatorial optimization problems which contain uncertainty along with non-determinism and dynamicity. These three properties call for app…
Breiman meets Bellman: Non-Greedy Decision Trees with MDPs
Hector Kohler, Riad Akrour, Philippe Preux
In supervised learning, decision trees are valued for their interpretability and performance. While greedy decision tree algorithms like CART remain widely used due to their comput…
AdaStop: adaptive statistical testing for sound comparisons of Deep RL agents
Timothée Mathieu, Riccardo Della Vecchia, Alena Shilova +4
Recently, the scientific community has questioned the statistical reproducibility of many empirical results, especially in the field of machine learning. To contribute to the resol…
A Machine of Few Words -- Interactive Speaker Recognition with Reinforcement Learning
Mathieu Seurin, Florian Strub, Philippe Preux +1
Speaker recognition is a well known and studied task in the speech processing domain. It has many applications, either for security or speaker adaptation of personal devices. In th…
Limits of Actor-Critic Algorithms for Decision Tree Policies Learning in IBMDPs
Hector Kohler, Riad Akrour, Philippe Preux
Interpretability of AI models allows for user safety checks to build trust in such AIs. In particular, Decision Trees (DTs) provide a global look at the learned model and transpare…
Adversarially Guided Actor-Critic
Yannis Flet-Berliac, Johan Ferret, Olivier Pietquin +2
Despite definite success in deep reinforcement learning problems, actor-critic algorithms are still confronted with sample inefficiency in complex environments, particularly in tas…
A generative model for sparse, evolving digraphs
Georgios Papoudakis, Philippe Preux, Martin Monperrus
Generating graphs that are similar to real ones is an open problem, while the similarity notion is quite elusive and hard to formalize. In this paper, we focus on sparse digraphs a…
There Is No Turning Back: A Self-Supervised Approach for Reversibility-Aware Reinforcement Learning
Nathan Grinsztajn, Johan Ferret, Olivier Pietquin +2
We propose to learn to distinguish reversible from irreversible actions for better informed decision-making in Reinforcement Learning (RL). From theoretical considerations, we show…
Only Relevant Information Matters: Filtering Out Noisy Samples to Boost RL
Yannis Flet-Berliac, Philippe Preux
In reinforcement learning, policy gradient algorithms optimize the policy directly and rely on sampling efficiently an environment. Nevertheless, while most sampling procedures are…
gym-DSSAT: a crop model turned into a Reinforcement Learning environment
Romain Gautron, Emilio J. Padrón, Philippe Preux +3
Addressing a real world sequential decision problem with Reinforcement Learning (RL) usually starts with the use of a simulated environment that mimics real conditions. We present…
Learning Value Functions in Deep Policy Gradients using Residual Variance
Yannis Flet-Berliac, Reda Ouhamma, Odalric-Ambrym Maillard +1
Policy gradient algorithms have proven to be successful in diverse decision making and control tasks. However, these methods suffer from high sample complexity and instability issu…
Towards a Research Community in Interpretable Reinforcement Learning: the InterpPol Workshop
Hector Kohler, Quentin Delfosse, Paul Festor +1
Embracing the pursuit of intrinsically explainable reinforcement learning raises crucial questions: what distinguishes explainability from interpretability? Should explainable and…
Optimal Interpretability-Performance Trade-off of Classification Trees with Black-Box Reinforcement Learning
Hector Kohler, Riad Akrour, Philippe Preux
Interpretability of AI models allows for user safety checks to build trust in these models. In particular, decision trees (DTs) provide a global view on the learned model and clear…
Soft Action Priors: Towards Robust Policy Transfer
Matheus Centa, Philippe Preux
Despite success in many challenging problems, reinforcement learning (RL) is still confronted with sample inefficiency, which can be mitigated by introducing prior knowledge to age…
A Learning Algorithm for Change Impact Prediction
Vincenzo Musco, Antonin Carette, Martin Monperrus +1
Change impact analysis consists in predicting the impact of a code change in a software application. In this paper, we take a learning perspective on change impact analysis and con…
Datum-Wise Classification: A Sequential Approach to Sparsity
Gabriel Dulac-Arnold, Ludovic Denoyer, Philippe Preux +1
We propose a novel classification technique whose aim is to select an appropriate representation for each datapoint, in contrast to the usual approach of selecting a representation…
MERL: Multi-Head Reinforcement Learning
Yannis Flet-Berliac, Philippe Preux
A common challenge in reinforcement learning is how to convert the agent's interactions with an environment into fast and robust learning. For instance, earlier work makes use of d…
A Large-Scale Study of Call Graph-based Impact Prediction using Mutation Testing
Vincenzo Musco, Martin Monperrus, Philippe Preux
In software engineering, impact analysis involves predicting the software elements (e.g., modules, classes, methods) potentially impacted by a change in the source code. Impact ana…
A Generalized Kernel Approach to Structured Output Learning
Hachem Kadri, Mohammad Ghavamzadeh, Philippe Preux
We study the problem of structured output learning from a regression perspective. We first provide a general formulation of the kernel dependency estimation (KDE) problem using ope…
Don't Do What Doesn't Matter: Intrinsic Motivation with Action Usefulness
Mathieu Seurin, Florian Strub, Philippe Preux +1
Sparse rewards are double-edged training signals in reinforcement learning: easy to design but hard to optimize. Intrinsic motivation guidances have thus been developed toward alle…
Functional Regularized Least Squares Classi cation with Operator-valued Kernels
Hachem Kadri, Asma Rabaoui, Philippe Preux +2
Although operator-valued kernels have recently received increasing interest in various machine learning and functional data analysis problems such as multi-task learning or functio…
Multiple Operator-valued Kernel Learning
Hachem Kadri, Alain Rakotomamonjy, Francis Bach +1
Positive definite operator-valued kernels generalize the well-known notion of reproducing kernels, and are naturally adapted to multi-output learning situations. This paper address…
Bandits attack function optimization
Philippe Preux, Rémi Munos, Michal Valko
We consider function optimization as a sequential decision making problem under budget constraint. This constraint limits the number of objective function evaluations allowed durin…
Interferometric Graph Transform for Community Labeling
Nathan Grinsztajn, Louis Leconte, Philippe Preux +1
We present a new approach for learning unsupervised node representations in community graphs. We significantly extend the Interferometric Graph Transform (IGT) to community labelin…
IDEQ -- Improving Diffusion Models for the Traveling Salesman Problem (TSP) by Leveraging the Structure of the Solution Space
Mickael Basson, Philippe Preux
We investigate diffusion models to solve the Traveling Salesman Problem. Building on the recent DIFUSCO and T2TCO approaches, we propose IDEQ. IDEQ improves the quality of the solu…
Interpretable and Editable Programmatic Tree Policies for Reinforcement Learning
Hector Kohler, Quentin Delfosse, Riad Akrour +2
Deep reinforcement learning agents are prone to goal misalignments. The black-box nature of their policies hinders the detection and correction of such misalignments, and the trust…
More Efficient Exploration with Symbolic Priors on Action Sequence Equivalences
Toby Johnstone, Nathan Grinsztajn, Johan Ferret +1
Incorporating prior knowledge in reinforcement learning algorithms is mainly an open question. Even when insights about the environment dynamics are available, reinforcement learni…