372 citations · 849 across the 97 of their papers we have counts for
41 papers · 1 filter
PlayPolicy: Dexterity from Supervised Play
Botao He, Yiming Dou, Yiannis Aloimonos +3
How can robots autonomously collect the contact-rich experience needed to learn dexterous manipulation? We present PlayPolicy: Dexterity from supervised play (DSP), a framework in…
RingNav: Task-Shaped Circular Memory for Visual Heading
Naitri Rajyaguru, Aryan Tajne, Stephen Jardim +2
Tiny robots have limited computation and memory, yet visual navigation systems often maintain more state than a task requires. We ask whether persistent memory can instead be shape…
What to Attend, What to Keep: Skill-Conditioned Visuotactile Representation with Progress-Guided Event Memory
Amir-Hossein Shahidzadeh, Seungjae Lee, Eadom Dessalene +5
Robotic manipulation integrates vision, touch, and language, whose importance shifts across stages: vision guides reaching, while touch, through its evolution over time, decides gr…
ForceBand: Learning Forceful Manipulation with sEMG
Botao He, Zhi Wang, Linna Kuang +8
Human demonstrations are a scalable data source for learning robot manipulation policies. However, common sources of human demonstration data, such as motion-capture trajectories a…
HumanEgo: Zero-Shot Robot Learning from Minutes of Human Egocentric Videos
Zhi Wang, Botao He, Kelin Yu +4
Human egocentric video captures rich manipulation demonstrations without any robot hardware, yet transferring these skills to robots remains challenging due to the embodiment gap b…
Monocular Reconstruction of Neural Tactile Fields
Pavan Mantripragada, Siddhanth Deshmukh, Eadom Dessalene +2
Robots operating in the real world must plan through environments that deform, yield, and reconfigure under contact, requiring interaction-aware 3D representations that extend beyo…
Artificial Microsaccade Compensation: Stable Vision for an Ornithopter
Levi Burner, Guido de Croon, Yiannis Aloimonos
Animals with foveated vision, including humans, experience microsaccades, small, rapid eye movements that they are not aware of. Inspired by this phenomenon, we develop a method fo…
Adversarial Game-Theoretic Algorithm for Dexterous Grasp Synthesis
Yu Chen, Botao He, Yuemin Mao +7
For many complex tasks, multi-finger robot hands are poised to revolutionize how we interact with the world, but reliably grasping objects remains a significant challenge. We focus…
EmbodiSwap for Zero-Shot Robot Imitation Learning
Eadom Dessalene, Pavan Mantripragada, Michael Maynord +1
We introduce EmbodiSwap - a method for producing photorealistic synthetic robot overlays over human video. We employ EmbodiSwap for zero-shot imitation learning, bridging the embod…
NavMoE: Hybrid Model- and Learning-based Traversability Estimation for Local Navigation via Mixture of Experts
Botao He, Amir Hossein Shahidzadeh, Yu Chen +8
This paper explores traversability estimation for robot navigation. A key bottleneck in traversability estimation lies in efficiently achieving reliable and robust predictions whil…
Object Pose Estimation through Dexterous Touch
Amir-Hossein Shahidzadeh, Jiyue Zhu, Kezhou Chen +4
Robust object pose estimation is essential for manipulation and interaction tasks in robotics, particularly in scenarios where visual data is limited or sensitive to lighting, occl…
DREAM: Domain-aware Reasoning for Efficient Autonomous Underwater Monitoring
Zhenqi Wu, Abhinav Modi, Angelos Mavrogiannis +6
The ocean is warming and acidifying, increasing the risk of mass mortality events for temperature-sensitive shellfish such as oysters. This motivates the development of long-term m…
NatSGLD: A Dataset with Speech, Gesture, Logic, and Demonstration for Robot Learning in Natural Human-Robot Interaction
Snehesh Shrestha, Yantian Zha, Saketh Banagiri +3
Recent advances in multimodal Human-Robot Interaction (HRI) datasets emphasize the integration of speech and gestures, allowing robots to absorb explicit knowledge and tacit unders…
FeelAnyForce: Estimating Contact Force Feedback from Tactile Sensation for Vision-Based Tactile Sensors
Amir-Hossein Shahidzadeh, Gabriele Caddeo, Koushik Alapati +3
In this paper, we tackle the problem of estimating 3D contact forces using vision-based tactile sensors. In particular, our goal is to estimate contact forces over a large range (u…
Extremum Seeking Controlled Wiggling for Tactile Insertion
Levi Burner, Pavan Mantripragada, Gabriele M. Caddeo +3
When humans perform complex insertion tasks such as pushing a cup into a cupboard, routing a cable, or putting a key in a lock, they wiggle the object and adapt the process through…
Embodied Visuomotor Representation
Levi Burner, Cornelia Fermüller, Yiannis Aloimonos
Imagine sitting at your desk, looking at objects on it. You do not know their exact distances from your eye in meters, but you can immediately reach out and touch them. Instead of…
Air-FAR: Fast and Adaptable Routing for Aerial Navigation in Large-scale Complex Unknown Environments
Botao He, Guofei Chen, Cornelia Fermuller +2
This paper presents a novel method for real-time 3D navigation in large-scale, complex environments using a hierarchical 3D visibility graph (V-graph). The proposed algorithm addre…
Discovering Object Attributes by Prompting Large Language Models with Perception-Action APIs
Angelos Mavrogiannis, Dehao Yuan, Yiannis Aloimonos
There has been a lot of interest in grounding natural language to physical entities through visual context. While Vision Language Models (VLMs) can ground linguistic instructions t…
ViewActive: Active viewpoint optimization from a single image
Jiayi Wu, Xiaomin Lin, Botao He +2
When observing objects, humans benefit from their spatial visualization and mental rotation ability to envision potential optimal viewpoints based on the current observation. This…
CodedVO: Coded Visual Odometry
Sachin Shah, Naitri Rajyaguru, Chahat Deep Singh +2
Autonomous robots often rely on monocular cameras for odometry estimation and navigation. However, the scale ambiguity problem presents a critical barrier to effective monocular vi…
Active Human Pose Estimation via an Autonomous UAV Agent
Jingxi Chen, Botao He, Chahat Deep Singh +2
One of the core activities of an active observer involves moving to secure a "better" view of the scene, where the definition of "better" is task-dependent. This paper focuses on t…
Microsaccade-inspired Event Camera for Robotics
Botao He, Ze Wang, Yuan Zhou +11
Neuromorphic vision sensors or event cameras have made the visual perception of extremely low reaction time possible, opening new avenues for high-dynamic robotics applications. Th…
Interactive-FAR:Interactive, Fast and Adaptable Routing for Navigation Among Movable Obstacles in Complex Unknown Environments
Botao He, Guofei Chen, Wenshan Wang +3
This paper introduces a real-time algorithm for navigating complex unknown environments cluttered with movable obstacles. Our algorithm achieves fast, adaptable routing by actively…
NatSGD: A Dataset with Speech, Gestures, and Demonstrations for Robot Learning in Natural Human-Robot Interaction
Snehesh Shrestha, Yantian Zha, Saketh Banagiri +3
Recent advancements in multimodal Human-Robot Interaction (HRI) datasets have highlighted the fusion of speech and gesture, expanding robots' capabilities to absorb explicit and im…