13 papers · 1 filter
HELP: Hierarchical Embodied Language Planner for Household Tasks
Alexandr V. Korchemnyi, Anatoly O. Onishchenko, Eva A. Bakaeva +2
Embodied agents tasked with complex scenarios, whether in real or simulated environments, rely heavily on robust planning capabilities. When instructions are formulated in natural…
LookPlanGraph: Embodied Instruction Following Method with VLM Graph Augmentation
Anatoly O. Onishchenko, Alexey K. Kovalev, Aleksandr I. Panov
Methods that use Large Language Models (LLM) as planners for embodied instruction following tasks have become widespread. To successfully complete tasks, the LLM must be grounded i…
Don't Blind Your VLA: Aligning Visual Representations for OOD Generalization
Nikita Kachaev, Mikhail Kolosov, Daniil Zelezetsky +2
The growing success of Vision-Language-Action (VLA) models stems from the promise that pretrained Vision-Language Models (VLMs) can endow agents with transferable world knowledge a…
A New Perspective on Transformers in Online Reinforcement Learning for Continuous Control
Nikita Kachaev, Daniil Zelezetsky, Egor Cherepanov +2
Despite their effectiveness and popularity in offline or model-based reinforcement learning (RL), transformers remain underexplored in online model-free RL due to their sensitivity…
ELMUR: External Layer Memory with Update/Rewrite for Long-Horizon RL Problems
Egor Cherepanov, Alexey K. Kovalev, Aleksandr I. Panov
Real-world robotic agents must act under partial observability and long horizons, where key cues may appear long before they affect decision making. However, most modern approaches…
Accelerating Transformers in Online RL
Daniil Zelezetsky, Alexey K. Kovalev, Aleksandr I. Panov
The appearance of transformer-based models in Reinforcement Learning (RL) has expanded the horizons of possibilities in robotics tasks, but it has simultaneously brought a wide ran…
Re:Frame -- Retrieving Experience From Associative Memory
Daniil Zelezetsky, Egor Cherepanov, Alexey K. Kovalev +1
Offline reinforcement learning (RL) often deals with suboptimal data when collecting large expert datasets is unavailable or impractical. This limitation makes it difficult for age…
Mind and Motion Aligned: A Joint Evaluation IsaacSim Benchmark for Task Planning and Low-Level Policies in Mobile Manipulation
Nikita Kachaev, Andrei Spiridonov, Andrey Gorodetsky +8
Benchmarks are crucial for evaluating progress in robotics and embodied AI. However, a significant gap exists between benchmarks designed for high-level language instruction follow…
Spatial Traces: Enhancing VLA Models with Spatial-Temporal Understanding
Maxim A. Patratskiy, Alexey K. Kovalev, Aleksandr I. Panov
Vision-Language-Action models have demonstrated remarkable capabilities in predicting agent movements within virtual environments and real-world scenarios based on visual observati…
VerifyLLM: LLM-Based Pre-Execution Task Plan Verification for Robots
Danil S. Grigorev, Alexey K. Kovalev, Aleksandr I. Panov
In the field of robotics, researchers face a critical challenge in ensuring reliable and efficient task planning. Verifying high-level task plans before execution significantly red…
LERa: Replanning with Visual Feedback in Instruction Following
Svyatoslav Pchelintsev, Maxim Patratskiy, Anatoly Onishchenko +7
Large Language Models are increasingly used in robotics for task planning, but their reliance on textual inputs limits their adaptability to real-world changes and failures. To add…
AmbiK: Dataset of Ambiguous Tasks in Kitchen Environment
Anastasiia Ivanova, Eva Bakaeva, Zoya Volovikova +2
As a part of an embodied agent, Large Language Models (LLMs) are typically used for behavior planning given natural language instructions from the user. However, dealing with ambig…
Memory, Benchmark & Robots: A Benchmark for Solving Complex Tasks with Reinforcement Learning
Egor Cherepanov, Nikita Kachaev, Alexey K. Kovalev +1
Memory is crucial for enabling agents to tackle complex tasks with temporal and spatial dependencies. While many reinforcement learning (RL) algorithms incorporate memory, the fiel…