J. Pajarinen

V-SHARE: Provable Convergence Guarantees for Advantage-based Algorithms

Kintan Saha, Hamed Masoudi, Xinyi Sheng +3

In reinforcement learning, advantage-based value-learning methods, unlike standard Q-learning, share information across actions by decomposing the action-value function into a stat…

SUN: Reaching for Novelty in Reinforcement Learning

Wenyan Yang, Arsenii Mustafin, Dominik Baumann +2

Exploration in reinforcement learning (RL) remains a fundamental challenge. Recent goal-conditioned RL strategies (which select goals to encourage broader state coverage) have show…

Momba: Network Modernization Improves Multi-Objective Reinforcement Learning

Adam Štafa, Santeri Heiskanen, Petr Novotný +1

Recent advances in deep reinforcement learning (RL) have shown that improving neural network architectures can yield substantial gains in sample efficiency and asymptotic performan…

Learning Multi-Timescale Abstractions for Hierarchical Combinatorial Planning

Vivienne Huiling Wang, Tinghuai Wang, Joni Pajarinen

The combination of exponentially large action spaces, stochastic dynamics, and long-horizon decision-making under limited resources makes Sequential Stochastic Combinatorial Optimi…

Temporal Consistency Improves Generalization in Contextual Offline Meta Reinforcement Learning

Mohammadreza Nakheai, Aidan Scannell, Kevin Luck +1

Offline meta-reinforcement learning seeks to learn a policy that generalizes to new related tasks online. Context-based methods infer a task representation from transition historie…

Pareto-Conditioned Diffusion Models for Offline Multi-Objective Optimization

Jatan Shrestha, Santeri Heiskanen, Kari Hepola +3

Multi-objective optimization (MOO) arises in many real-world applications where trade-offs between competing objectives must be carefully balanced. In the offline setting, where on…

Sparsely Supervised Diffusion

Wenshuai Zhao, Zhiyuan Li, Yi Zhao +5

Diffusion models have shown remarkable success across a wide range of generative tasks. However, they often suffer from spatially inconsistent generation, arguably due to the inher…

GCHR : Goal-Conditioned Hindsight Regularization for Sample-Efficient Reinforcement Learning

Xing Lei, Wenyan Yang, Kaiqiang Ke +4

Goal-conditioned reinforcement learning (GCRL) with sparse rewards remains a fundamental challenge in reinforcement learning. While hindsight experience replay (HER) has shown prom…

Reachability Weighted Offline Goal-conditioned Resampling

Wenyan Yang, Joni Pajarinen

Offline goal-conditioned reinforcement learning (RL) relies on fixed datasets where many potential goals share the same state and action spaces. However, these potential goals are…

Hierarchical Reinforcement Learning with Uncertainty-Guided Diffusional Subgoals

Vivienne Huiling Wang, Tinghuai Wang, Joni Pajarinen

Hierarchical reinforcement learning (HRL) learns to make decisions on multiple levels of temporal abstraction. A key challenge in HRL is that the low-level policy changes over time…

Discrete Codebook World Models for Continuous Control

Aidan Scannell, Mohammadreza Nakhaei, Kalle Kujanpää +4

In reinforcement learning (RL), world models serve as internal simulators, enabling agents to predict environment dynamics and future outcomes in order to make informed decisions.…

Efficient Reinforcement Learning by Guiding World Models with Non-Curated Data

Yi Zhao, Aidan Scannell, Wenshuai Zhao +7

Leveraging offline data is a promising way to improve the sample efficiency of online reinforcement learning (RL). This paper expands the pool of usable data for offline-to-online…

Entropy Regularized Task Representation Learning for Offline Meta-Reinforcement Learning

Mohammadreza Nakhaei, Aidan Scannell, Joni Pajarinen

Offline meta-reinforcement learning aims to equip agents with the ability to rapidly adapt to new tasks by training on data from a set of different tasks. Context-based approaches…

ROER: Regularized Optimal Experience Replay

Changling Li, Zhang-Wei Hong, Pulkit Agrawal +2

Experience replay serves as a key component in the success of online reinforcement learning (RL). Prioritized experience replay (PER) reweights experiences by the temporal differen…

Probabilistic Subgoal Representations for Hierarchical Reinforcement learning

Vivienne Huiling Wang, Tinghuai Wang, Wenyan Yang +2

In goal-conditioned hierarchical reinforcement learning (HRL), a high-level policy specifies a subgoal for the low-level policy to reach. Effective HRL hinges on a suitable subgoal…

Residual Learning and Context Encoding for Adaptive Offline-to-Online Reinforcement Learning

Mohammadreza Nakhaei, Aidan Scannell, Joni Pajarinen

Offline reinforcement learning (RL) allows learning sequential behavior from fixed datasets. Since offline datasets do not cover all possible situations, many methods collect addit…