12 papers
When Plans Change, Remember: Replanning-Guided Visual Memory for Vision-Language-Action Models
Victoria Smelova, Daniil Kazachkov, Georgiy Kazantsev +6
Long-horizon, multi-stage Vision-Language-Action (VLA) policies often require visual information that is no longer present in the current observation. Retaining the complete observ…
MIKASA-Robo-VLA: Benchmarking Memory in VLA Models for Long-Horizon Manipulation
Egor Cherepanov, Nikita Kachaev, Aleksandr I. Panov +1
Vision-language-action policies often see only one or a few recent frames, which makes it difficult to evaluate how they use information that disappears during a task. We introduce…
ALER: Adaptive Learnable Experience Rewriting for Reinforcement Learning
Oleg Shchendrigin, Egor Cherepanov, Aleksandr I. Panov +1
In partially observable reinforcement learning (RL), a later observation can make stored information obsolete or change what it implies for the next decision. Memory architectures…
VLA: On Recurrent Memory for Partially Observable Manipulation in VLA Models
Egor Cherepanov, Nikita Kachaev, Daniil Zelezetsky +6
Vision-language-action (VLA) models predict chunks of future actions from the current observation, an assumption that fails under partial observability, where decisions depend on i…
MARL-GPT: Foundation Model for Multi-Agent Reinforcement Learning
Maria Nesterova, Mikhail Kolosov, Anton Andreychuk +6
Recent advances in multi-agent reinforcement learning (MARL) have demonstrated success in numerous challenging domains and environments, but typically require specialized models fo…
Memory Retention Is Not Enough to Master Memory Tasks in Reinforcement Learning
Oleg Shchendrigin, Egor Cherepanov, Alexey K. Kovalev +1
Effective decision-making in the real world depends on memory that is both stable and adaptive: environments change over time, and agents must retain relevant information over long…
KAGE-Bench: Fast Known-Axis Visual Generalization Evaluation for Reinforcement Learning
Egor Cherepanov, Daniil Zelezetsky, Alexey K. Kovalev +1
Pixel-based reinforcement learning agents often fail under purely visual distribution shift even when latent dynamics and rewards are unchanged, but existing benchmarks entangle mu…
A New Perspective on Transformers in Online Reinforcement Learning for Continuous Control
Nikita Kachaev, Daniil Zelezetsky, Egor Cherepanov +2
Despite their effectiveness and popularity in offline or model-based reinforcement learning (RL), transformers remain underexplored in online model-free RL due to their sensitivity…
ELMUR: External Layer Memory with Update/Rewrite for Long-Horizon RL Problems
Egor Cherepanov, Alexey K. Kovalev, Aleksandr I. Panov
Real-world robotic agents must act under partial observability and long horizons, where key cues may appear long before they affect decision making. However, most modern approaches…
Re:Frame -- Retrieving Experience From Associative Memory
Daniil Zelezetsky, Egor Cherepanov, Alexey K. Kovalev +1
Offline reinforcement learning (RL) often deals with suboptimal data when collecting large expert datasets is unavailable or impractical. This limitation makes it difficult for age…
Memory, Benchmark & Robots: A Benchmark for Solving Complex Tasks with Reinforcement Learning
Egor Cherepanov, Nikita Kachaev, Alexey K. Kovalev +1
Memory is crucial for enabling agents to tackle complex tasks with temporal and spatial dependencies. While many reinforcement learning (RL) algorithms incorporate memory, the fiel…
Unraveling the Complexity of Memory in RL Agents: an Approach for Classification and Evaluation
Egor Cherepanov, Nikita Kachaev, Artem Zholus +2
The incorporation of memory into agents is essential for numerous tasks within the domain of Reinforcement Learning (RL). In particular, memory is paramount for tasks that require…