91 citations · 167 across the 19 of their papers we have counts for
22 papers
Fine-Grained Emotion Detection on GoEmotions: Experimental Comparison of Classical Machine Learning, BiLSTM, and Transformer Models
Ani Harutyunyan, Sachin Kumar
Fine-grained emotion recognition is a challenging multi-label NLP task due to label overlap and class imbalance. In this work, we benchmark three modeling families on the GoEmotion…
Plasticity as the Mirror of Empowerment
David Abel, Michael Bowling, André Barreto +13
Agents are minimally entities that are influenced by their past observations and act to influence future observations. This latter capacity is captured by empowerment, which has se…
Agency Is Frame-Dependent
David Abel, André Barreto, Michael Bowling +13
Agency is a system's capacity to steer outcomes toward a goal, and is a central topic of study across biology, philosophy, cognitive science, and artificial intelligence. Determini…
Three Dogmas of Reinforcement Learning
David Abel, Mark K. Ho, Anna Harutyunyan
Modern reinforcement learning has been conditioned by at least three dogmas. The first is the environment spotlight, which refers to our tendency to focus on modeling environments…
Bootstrapped Representations in Reinforcement Learning
Charline Le Lan, Stephen Tu, Mark Rowland +4
In reinforcement learning (RL), state representations are key to dealing with large or continuous state spaces. While one of the promises of deep learning algorithms is to automati…
DoMo-AC: Doubly Multi-step Off-policy Actor-Critic Algorithm
Yunhao Tang, Tadashi Kozuno, Mark Rowland +4
Multi-step learning applies lookahead over multiple time steps and has proved valuable in policy evaluation settings. However, in the optimal control case, the impact of multi-step…
An Analysis of Quantile Temporal-Difference Learning
Mark Rowland, Rémi Munos, Mohammad Gheshlaghi Azar +6
We analyse quantile temporal-difference learning (QTD), a distributional reinforcement learning algorithm that has proven to be a key component in several successful large-scale ap…
On the Expressivity of Markov Reward
David Abel, Will Dabney, Anna Harutyunyan +4
Reward is the driving force for reinforcement-learning agents. This paper is dedicated to understanding the expressivity of reward as a way to capture tasks that we would want an a…
Dynamical-System Model Predicts When Social Learners Impair Collective Performance
Vicky Chuqiao Yang, Mirta Galesic, Harvey McGuinness +1
A key question concerning collective decisions is whether a social system can settle on the best available option when some members learn from others instead of evaluating the opti…
Useful Policy Invariant Shaping from Arbitrary Advice
Paniz Behboudian, Yash Satsangi, Matthew E. Taylor +2
Reinforcement learning is a powerful learning paradigm in which agents can learn to maximize sparse and delayed reward signals. Although RL has had many impressive successes in com…
Counterfactual Credit Assignment in Model-Free Reinforcement Learning
Thomas Mesnard, Théophane Weber, Fabio Viola +11
Credit assignment in reinforcement learning is the problem of measuring an action's influence on future rewards. In particular, this requires separating skill from luck, i.e. disen…
Hindsight Credit Assignment
Anna Harutyunyan, Will Dabney, Thomas Mesnard +8
We consider the problem of efficient credit assignment in reinforcement learning. In order to efficiently and meaningfully utilize new data, we propose to explicitly assign credit…
Conditional Importance Sampling for Off-Policy Learning
Mark Rowland, Anna Harutyunyan, Hado van Hasselt +4
The principal contribution of this paper is a conceptual framework for off-policy reinforcement learning, based on conditional expectations of importance sampling ratios. This fram…
The Termination Critic
Anna Harutyunyan, Will Dabney, Diana Borsa +3
In this work, we consider the problem of autonomously discovering behavioral abstractions, or options, for reinforcement learning agents. We propose an algorithm that focuses on th…
Learning with Options that Terminate Off-Policy
Anna Harutyunyan, Peter Vrancx, Pierre-Luc Bacon +2
A temporally abstract action, or an option, is specified by a policy and a termination condition: the policy guides option behavior, and the termination condition roughly determine…
Reinforcement Learning in POMDPs with Memoryless Options and Option-Observation Initiation Sets
Denis Steckelmacher, Diederik M. Roijers, Anna Harutyunyan +3
Many real-world reinforcement learning problems have a hierarchical nature, and often exhibit some degree of partial observability. While hierarchy and partial observability are us…
Safe and Efficient Off-Policy Reinforcement Learning
Rémi Munos, Tom Stepleton, Anna Harutyunyan +1
In this work, we take a fresh look at some old and new algorithms for off-policy, return-based reinforcement learning. Expressing these in a common form, we derive a novel algorith…
Q() with Off-Policy Corrections
Anna Harutyunyan, Marc G. Bellemare, Tom Stepleton +1
We propose and analyze an alternate approach to off-policy multi-step temporal difference learning, in which off-policy returns are corrected with the current Q-function in terms o…
Off-Policy Reward Shaping with Ensembles
Anna Harutyunyan, Tim Brys, Peter Vrancx +1
Potential-based reward shaping (PBRS) is an effective and popular technique to speed up reinforcement learning by leveraging domain knowledge. While PBRS is proven to always preser…
Off-Policy Shaping Ensembles in Reinforcement Learning
Anna Harutyunyan, Tim Brys, Peter Vrancx +1
Recent advances of gradient temporal-difference methods allow to learn off-policy multiple value functions in parallel with- out sacrificing convergence guarantees or computational…
Boundary-to-boundary flows in planar graphs
Glencora Borradaile, Anna Harutyunyan
We give an iterative algorithm for finding the maximum flow between a set of sources and sinks that lie on the boundary of a planar graph. Our algorithm uses only O(n) queries to s…
Maximum st-flow in directed planar graphs via shortest paths
Glencora Borradaile, Anna Harutyunyan
Minimum cuts have been closely related to shortest paths in planar graphs via planar duality - so long as the graphs are undirected. Even maximum flows are closely related to short…