21 papers
Rationale Distribution Distillation via Flow Matching for Distributional Continuous Latent Reasoning
Mingyeong Kim, Gyusang Cho, Juho Lee +1
Sampling multiple chain-of-thought rationales from a language model can uncover correct solutions missed by a single attempt. However, these rationales are typically lengthy, makin…
World Modeling through Spectral Alignment
Holger Molin, William Peng, Marco Bagatella +1
How do we tell a world model what information to preserve? We study this question through the lens of spectral representation learning. We introduce SpecWM, a JEPA framework that e…
Deep Epistemic Value Functions for Optimistic Exploration
Leander Diaz-Bone, Marco Bagatella, Jonas Hübotter +1
Principled exploration in reinforcement learning requires an agent to quantify its epistemic uncertainty and act to resolve it. Uncertainty over the value function provides a natur…
Three Steps at a Time: Learning Representations from Action Sequences in Contrastive RL
Michal Korniak, Kamil Dybek, Benjamin Eysenbach +2
While self-supervised approaches to reinforcement learning have achieved strong results by learning representations of states and actions, a key open question is the time scale ove…
Uncertainty Quantification for Flow-Based Generalist Robot Policies
Ralf Römer, Maximilian Seeliger, Saida Liu +5
Generalist robot policies, such as vision-language-action models (VLAs) and world-action models (WAMs), combine powerful pretrained backbones with expressive generative action head…
Majority Voting for Code Generation
Tim Launer, Jonas Hübotter, Marco Bagatella +2
We investigate Functional Majority Voting (FMV), a method based on functional consensus for code generation with Large Language Models, which identifies a representative solution f…
Reinforcement Learning via Self-Distillation
Jonas Hübotter, Frederike Lübeck, Lejs Behric +8
Large language models are increasingly post-trained with reinforcement learning in verifiable domains such as code and math. Yet, current methods for reinforcement learning with ve…
Soft Forward-Backward Representations for Zero-shot Reinforcement Learning with General Utilities
Marco Bagatella, Thomas Rupf, Georg Martius +1
Recent advancements in zero-shot reinforcement learning (RL) have facilitated the extraction of diverse behaviors from unlabeled, offline data sources. In particular, forward-backw…
Optimistic Task Inference for Behavior Foundation Models
Thomas Rupf, Marco Bagatella, Marin Vlastelica +1
Behavior Foundation Models (BFMs) are capable of retrieving high-performing policy for any reward function specified directly at test-time, commonly referred to as zero-shot reinfo…
TD-JEPA: Latent-predictive Representations for Zero-Shot Reinforcement Learning
Marco Bagatella, Matteo Pirotta, Ahmed Touati +2
Latent prediction--where agents learn by predicting their own latents--has emerged as a powerful paradigm for training general representations in machine learning. In reinforcement…
Test-time Offline Reinforcement Learning on Goal-related Experience
Marco Bagatella, Mert Albaba, Jonas Hübotter +2
Foundation models compress a large amount of information in a single, large neural network, which can then be queried for individual tasks. There are strong parallels between this…
DISCOVER: Automated Curricula for Sparse-Reward Reinforcement Learning
Leander Diaz-Bone, Marco Bagatella, Jonas Hübotter +1
Sparse-reward reinforcement learning (RL) can model a wide range of highly complex tasks. Solving sparse-reward tasks is RL's core premise, requiring efficient exploration coupled…
Problem Space Transformations for Out-of-Distribution Generalisation in Behavioural Cloning
Kiran Doshi, Marco Bagatella, Stelian Coros
The combination of behavioural cloning and neural networks has driven significant progress in robotic manipulation. As these algorithms may require a large number of demonstrations…
Zero-Shot Offline Imitation Learning via Optimal Transport
Thomas Rupf, Marco Bagatella, Nico Gürtler +2
Zero-shot imitation learning algorithms hold the promise of reproducing unseen behavior from as little as a single demonstration at test time. Existing practical approaches view th…
Active Fine-Tuning of Multi-Task Policies
Marco Bagatella, Jonas Hübotter, Georg Martius +1
Pre-trained generalist policies are rapidly gaining relevance in robot learning due to their promise of fast adaptation to novel, in-domain tasks. This adaptation often relies on c…
Directed Exploration in Reinforcement Learning from Linear Temporal Logic
Marco Bagatella, Andreas Krause, Georg Martius
Linear temporal logic (LTL) is a powerful language for task specification in reinforcement learning, as it allows describing objectives beyond the expressivity of conventional disc…
Causal Action Influence Aware Counterfactual Data Augmentation
Núria Armengol Urpí, Marco Bagatella, Marin Vlastelica +1
Offline data are both valuable and practical resources for teaching robots complex behaviors. Ideally, learning agents should not be constrained by the scarcity of available demons…
Goal-conditioned Offline Planning from Curious Exploration
Marco Bagatella, Georg Martius
Curiosity has established itself as a powerful exploration strategy in deep reinforcement learning. Notably, leveraging expected future novelty as intrinsic motivation has been sho…
Efficient Learning of High Level Plans from Play
Núria Armengol Urpí, Marco Bagatella, Otmar Hilliges +2
Real-world robotic manipulation tasks remain an elusive challenge, since they involve both fine-grained environment interaction, as well as the ability to plan for long-horizon goa…
SFP: State-free Priors for Exploration in Off-Policy Reinforcement Learning
Marco Bagatella, Sammy Christen, Otmar Hilliges
Efficient exploration is a crucial challenge in deep reinforcement learning. Several methods, such as behavioral priors, are able to leverage offline data in order to efficiently a…
Planning from Pixels in Environments with Combinatorially Hard Search Spaces
Marco Bagatella, Mirek Olšák, Michal Rolínek +1
The ability to form complex plans based on raw visual input is a litmus test for current capabilities of artificial intelligence, as it requires a seamless combination of visual pr…