514 citations · 2.1k across the 35 of their papers we have counts for
39 papers
Learning Robot Policies from Sparse Success Signals via STL-Guided Stein Variational Policy Gradient
Hongrui Zheng, Cristian Ioan Vasile, Antonio Loquercio +1
Learning robot policies for tasks with sparse success signals is challenging when completion depends on coordinated actions, precise contact outcomes, or satisfying several conditi…
AI Coaching for Accelerating Human Skill Development with Reinforcement Learning
Wei Wang, Enlin Gu, Antonio Loquercio +2
AI copilots can substantially boost human performance through shared control, but excessive assistance can induce over-reliance and skill atrophy. This paper studies how an embodie…
RAVEN: Long-Horizon Reasoning & Navigation with a Visuo-Spatio-Temporal Memory
Yixun Hu, Zhicheng Zheng, Lihan Zha +5
Long-term robot deployment requires a compact and scalable memory that preserves fine-grained visual semantics, grounds observations in space and time, and enables efficient storag…
Reinforcement Learning for Flow-Matching Policies with Density Transport
Boshu Lei, Kostas Daniilidis, Antonio Loquercio
We present an online reinforcement learning (RL) algorithm for fine-tuning flow-matching policies in continuous-control problems. Our key insight is to view RL-based policy improve…
RoSHI: A Versatile Robot-oriented Suit for Human Data In-the-Wild
Wenjing Margaret Mao, Jefferson Ng, Luyang Hu +2
Scaling up robot learning will likely require human data containing rich and long-horizon interactions in the wild. Existing approaches for collecting such data trade off portabili…
Efficient and Reliable Teleoperation through Real-to-Sim-to-Real Shared Autonomy
Shuo Sha, Yixuan Wang, Binghao Huang +2
Fine-grained, contact-rich teleoperation remains slow, error-prone, and unreliable in real-world manipulation tasks, even for experienced operators. Shared autonomy offers a promis…
Agile Flight Emerges from Multi-Agent Competitive Racing
Vineet Pasumarti, Lorenzo Bianchi, Antonio Loquercio
Through multi-agent competition and the sparse high-level objective of winning a race, we find that both agile flight (e.g., high-speed motion pushing the platform to its physical…
Deep Sensorimotor Control by Imitating Predictive Models of Human Motion
Himanshu Gaurav Singh, Pieter Abbeel, Jitendra Malik +1
As the embodiment gap between a robot and a human narrows, new opportunities arise to leverage datasets of humans interacting with their surroundings for robot learning. We propose…
Hearing Hands: Generating Sounds from Physical Interactions in 3D Scenes
Yiming Dou, Wonseok Oh, Yuqing Luo +2
We study the problem of making 3D scene reconstructions interactive by asking the following question: can we predict the sounds of human hands physically interacting with a scene?…
End-to-End Navigation with Vision Language Models: Transforming Spatial Reasoning into Question-Answering
Dylan Goetting, Himanshu Gaurav Singh, Antonio Loquercio
We present VLMnav, an embodied framework to transform a Vision-Language Model (VLM) into an end-to-end navigation policy. In contrast to prior work, we do not rely on a separation…
Hand-Object Interaction Pretraining from Videos
Himanshu Gaurav Singh, Antonio Loquercio, Carmelo Sferrazza +4
We present an approach to learn general robot manipulation priors from 3D hand-object interaction trajectories. We build a framework to use in-the-wild videos to generate sensorimo…
A Learning-based Quadcopter Controller with Extreme Adaptation
Dingqi Zhang, Antonio Loquercio, Jerry Tang +3
This paper introduces a learning-based low-level controller for quadcopters, which adaptively controls quadcopters with significant variations in mass, size, and actuator capabilit…
Tactile-Augmented Radiance Fields
Yiming Dou, Fengyu Yang, Yi Liu +2
We present a scene representation, which we call a tactile-augmented radiance field (TaRF), that brings vision and touch into a shared 3D space. This representation can be used to…
EgoPet: Egomotion and Interaction Data from an Animal's Perspective
Amir Bar, Arya Bakhtiar, Danny Tran +5
Animals perceive the world to plan their actions and interact with other agents to accomplish complex tasks, demonstrating capabilities that are still unmatched by AI systems. To a…
Conformal Policy Learning for Sensorimotor Control Under Distribution Shifts
Huang Huang, Satvik Sharma, Antonio Loquercio +3
This paper focuses on the problem of detecting and reacting to changes in the distribution of a sensorimotor controller's observables. The key idea is the design of switching polic…
A Remote Sim2real Aerial Competition: Fostering Reproducibility and Solutions' Diversity in Robotics Challenges
Spencer Teetaert, Wenda Zhao, Niu Xinyuan +21
Shared benchmark problems have historically been a fundamental driver of progress for scientific communities. In the context of academic conferences, competitions offer the opportu…
Learning Vision-based Pursuit-Evasion Robot Policies
Andrea Bajcsy, Antonio Loquercio, Ashish Kumar +1
Learning strategic robot behavior -- like that required in pursuit-evasion interactions -- under real-world constraints is extremely challenging. It requires exploiting the dynamic…
Agilicious: Open-Source and Open-Hardware Agile Quadrotor for Vision-Based Flight
Philipp Foehn, Elia Kaufmann, Angel Romero +8
Autonomous, agile quadrotor flight raises fundamental challenges for robotics research in terms of perception, planning, learning, and control. A versatile and standardized platfor…
Manipulator as a Tail: Promoting Dynamic Stability for Legged Locomotion
Huang Huang, Antonio Loquercio, Ashish Kumar +3
For locomotion, is an arm on a legged robot a liability or an asset for locomotion? Biological systems evolved additional limbs beyond legs that facilitates postural control. This…
Autonomous Drone Racing: A Survey
Drew Hanover, Antonio Loquercio, Leonard Bauersfeld +6
Over the last decade, the use of autonomous drone systems for surveying, search and rescue, or last-mile delivery has increased exponentially. With the rise of these applications c…
Learning Visual Locomotion with Cross-Modal Supervision
Antonio Loquercio, Ashish Kumar, Jitendra Malik
In this work, we show how to learn a visual walking policy that only uses a monocular RGB camera and proprioception. Since simulating RGB is hard, we necessarily have to learn visi…
Training Efficient Controllers via Analytic Policy Gradient
Nina Wiedemann, Valentin Wüest, Antonio Loquercio +3
Control design for robotic systems is complex and often requires solving an optimization to follow a trajectory accurately. Online optimization approaches like Model Predictive Con…
Learning a Single Near-hover Position Controller for Vastly Different Quadcopters
Dingqi Zhang, Antonio Loquercio, Xiangyu Wu +3
This paper proposes an adaptive near-hover position controller for quadcopters, which can be deployed to quadcopters of very different mass, size and motor constants, and also show…
Visual Attention Prediction Improves Performance of Autonomous Drone Racing Agents
Christian Pfeiffer, Simon Wengeler, Antonio Loquercio +1
Humans race drones faster than neural networks trained for end-to-end autonomous flight. This may be related to the ability of human pilots to select task-relevant visual informati…