9 citations · 27 across the 23 of their papers we have counts for
24 papers
Rotate the Loop: Blockwise Conformality for Looped Transformers
Alexey Zemtsov, Albina Klepach, Vadim Lomshakov +5
Looped transformers reuse one shared block across many recurrent steps, so depth becomes a quantity that can be spent at runtime rather than a stack of distinct layers. Those extra…
Dual Advantage Fields
Alexey Zemtsov, Maxim Bobrin, Alexander Nikulin +5
Offline goal-conditioned reinforcement learning requires both long-horizon reachability estimates and local action comparisons. Dual goal representations provide value fields that…
Vintix II: Decision Pre-Trained Transformer is a Scalable In-Context Reinforcement Learner
Andrei Polubarov, Lyubaykin Nikita, Alexander Derevyagin +11
Recent progress in in-context reinforcement learning (ICRL) has demonstrated its potential for training generalist agents that can acquire new tasks directly at inference. Algorith…
Vision-Language Models Unlock Task-Centric Latent Actions
Alexander Nikulin, Ilya Zisman, Albina Klepach +5
Latent Action Models (LAMs) have rapidly gained traction as an important component in the pre-training pipelines of leading Vision-Language-Action models. However, they fail when o…
NinA: Normalizing Flows in Action. Training VLA Models with Normalizing Flows
Denis Tarasov, Alexander Nikulin, Ilya Zisman +5
Recent advances in Vision-Language-Action (VLA) models have established a two-component architecture, where a pre-trained Vision-Language Model (VLM) encodes visual observations an…
cadrille: Multi-modal CAD Reconstruction with Reinforcement Learning
Maksim Kolodiazhnyi, Denis Tarasov, Dmitrii Zhemchuzhnikov +6
Computer-Aided Design (CAD) plays a central role in engineering and manufacturing, making it possible to create precise and editable 3D models. Using a variety of sensor or user-pr…
Zero-Shot Adaptation of Behavioral Foundation Models to Unseen Dynamics
Maksim Bobrin, Ilya Zisman, Alexander Nikulin +2
Behavioral Foundation Models (BFMs) proved successful in producing policies for arbitrary tasks in a zero-shot manner, requiring no test-time training or task-specific fine-tuning.…
Yes, Q-learning Helps Offline In-Context RL
Denis Tarasov, Alexander Nikulin, Ilya Zisman +6
Existing offline in-context reinforcement learning (ICRL) methods have predominantly relied on supervised training objectives, which are known to have limitations in offline RL set…
Object-Centric Latent Action Learning
Albina Klepach, Alexander Nikulin, Ilya Zisman +6
Leveraging vast amounts of unlabeled internet video data for embodied AI is currently bottlenecked by the lack of action labels and the presence of action-correlated visual distrac…
Latent Action Learning Requires Supervision in the Presence of Distractors
Alexander Nikulin, Ilya Zisman, Denis Tarasov +4
Recently, latent action learning, pioneered by Latent Action Policies (LAPO), have shown remarkable pre-training efficiency on observation-only data, offering potential for leverag…
Vintix: Action Model via In-Context Reinforcement Learning
Andrey Polubarov, Nikita Lyubaykin, Alexander Derevyagin +4
In-Context Reinforcement Learning (ICRL) represents a promising paradigm for developing generalist agents that learn at inference time through trial-and-error interactions, analogo…
N-Gram Induction Heads for In-Context RL: Improving Stability and Reducing Data Needs
Ilya Zisman, Alexander Nikulin, Viacheslav Sinii +5
In-context learning allows models like transformers to adapt to new tasks from a few examples without updating their weights, a desirable trait for reinforcement learning (RL). How…
XLand-100B: A Large-Scale Multi-Task Dataset for In-Context Reinforcement Learning
Alexander Nikulin, Ilya Zisman, Alexey Zemtsov +1
Following the success of the in-context learning paradigm in large-scale language and computer vision models, the recently emerging field of in-context reinforcement learning is ex…
Open RL Benchmark: Comprehensive Tracked Experiments for Reinforcement Learning
Shengyi Huang, Quentin Gallouédec, Florian Felten +30
In many Reinforcement Learning (RL) papers, learning curves are useful indicators to measure the effectiveness of RL algorithms. However, the complete raw data of the learning curv…
In-Context Reinforcement Learning for Variable Action Spaces
Viacheslav Sinii, Alexander Nikulin, Vladislav Kurenkov +2
Recently, it has been shown that transformers pre-trained on diverse datasets with multi-episode contexts can generalize to new reinforcement learning tasks in-context. A key limit…
Emergence of In-Context Reinforcement Learning from Noise Distillation
Ilya Zisman, Vladislav Kurenkov, Alexander Nikulin +2
Recently, extensive studies in Reinforcement Learning have been carried out on the ability of transformers to adapt in-context to various environments and tasks. Current in-context…
XLand-MiniGrid: Scalable Meta-Reinforcement Learning Environments in JAX
Alexander Nikulin, Vladislav Kurenkov, Ilya Zisman +3
Inspired by the diversity and depth of XLand and the simplicity and minimalism of MiniGrid, we present XLand-MiniGrid, a suite of tools and grid-world environments for meta-reinfor…
Katakomba: Tools and Benchmarks for Data-Driven NetHack
Vladislav Kurenkov, Alexander Nikulin, Denis Tarasov +1
NetHack is known as the frontier of reinforcement learning research where learning-based methods still need to catch up to rule-based solutions. One of the promising directions for…
Revisiting the Minimalist Approach to Offline Reinforcement Learning
Denis Tarasov, Vladislav Kurenkov, Alexander Nikulin +1
Recent years have witnessed significant advancements in offline reinforcement learning (RL), resulting in the development of numerous algorithms with varying degrees of complexity.…
Anti-Exploration by Random Network Distillation
Alexander Nikulin, Vladislav Kurenkov, Denis Tarasov +1
Despite the success of Random Network Distillation (RND) in various domains, it was shown as not discriminative enough to be used as an uncertainty estimator for penalizing out-of-…
Let Offline RL Flow: Training Conservative Agents in the Latent Space of Normalizing Flows
Dmitriy Akimov, Vladislav Kurenkov, Alexander Nikulin +2
Offline reinforcement learning aims to train a policy on a pre-recorded and fixed dataset without any additional environment interactions. There are two major challenges in this se…
Q-Ensemble for Offline RL: Don't Scale the Ensemble, Scale the Batch Size
Alexander Nikulin, Vladislav Kurenkov, Denis Tarasov +2
Training large neural networks is known to be time-consuming, with the learning duration taking days or even weeks. To address this problem, large-batch optimization was introduced…
CORL: Research-oriented Deep Offline Reinforcement Learning Library
Denis Tarasov, Alexander Nikulin, Dmitry Akimov +2
CORL is an open-source library that provides thoroughly benchmarked single-file implementations of both deep offline and offline-to-online reinforcement learning algorithms. It emp…
MineRL Diamond 2021 Competition: Overview, Results, and Lessons Learned
Anssi Kanervisto, Stephanie Milani, Karolis Ramanauskas +19
Reinforcement learning competitions advance the field by providing appropriate scope and support to develop solutions toward a specific problem. To promote the development of more…