372 citations · 849 across the 97 of their papers we have counts for
107 papers
Dexterity from Supervised Play
Botao He, Yiming Dou, Yiannis Aloimonos +3
How can robots autonomously collect the contact-rich experience needed to learn dexterous manipulation? We present DSP: Dexterity from supervised play, a framework in which a robot…
RingNav: Task-Shaped Circular Memory for Visual Heading
Naitri Rajyaguru, Aryan Tajne, Stephen Jardim +2
Tiny robots have limited computation and memory, yet visual navigation systems often maintain more state than a task requires. We ask whether persistent memory can instead be shape…
FreeCHROMA: Generative Chroma Keying from Unaligned Background References
Jiayi Wu, Tianfu Wang, Rui Zhao +6
In film production, chroma keying composites an approved foreground performance into a new environment, but conventional workflows require coordinated background plates and dedicat…
What to Attend, What to Keep: Skill-Conditioned Visuotactile Representation with Progress-Guided Event Memory
Amir-Hossein Shahidzadeh, Seungjae Lee, Eadom Dessalene +5
Robotic manipulation integrates vision, touch, and language, whose importance shifts across stages: vision guides reaching, while touch, through its evolution over time, decides gr…
WaterGen: Decoupling Scene and Medium in Underwater Image Generation
Jiayi Wu, Tianfu Wang, Tianyi Xiong +6
Underwater computer vision tasks, such as detection, restoration, and segmentation, are limited by the scarcity of large-scale and diverse training data. We introduce WaterGen, a m…
ForceBand: Learning Forceful Manipulation with sEMG
Botao He, Zhi Wang, Linna Kuang +8
Human demonstrations are a scalable data source for learning robot manipulation policies. However, common sources of human demonstration data, such as motion-capture trajectories a…
Real2SAM2Real: Generative 3D Caches as Complementary Context for Video Diffusion
Jiayi Wu, Haoming Cai, Cornelia Fermuller +2
Video Diffusion Models (VDMs) synthesize high-quality videos, but precise camera and motion control remains challenging when large viewpoint shifts, changing occlusions, or complex…
HumanEgo: Zero-Shot Robot Learning from Minutes of Human Egocentric Videos
Zhi Wang, Botao He, Kelin Yu +4
Human egocentric video captures rich manipulation demonstrations without any robot hardware, yet transferring these skills to robots remains challenging due to the embodiment gap b…
NeuroAI and Beyond: Bridging Between Advances in Neuroscience and Artificial Intelligence
Anthony Zador, Jean-Marc Fellous, Terrence Sejnowski +28
Neuroscience and Artificial Intelligence (AI) have made impressive progress in recent years but remain only loosely interconnected. Based on a workshop convened by the National Sci…
FEEL (Force-Enhanced Egocentric Learning): A Dataset for Physical Action Understanding
Eadom Dessalene, Botao He, Michael Maynord +5
We introduce FEEL (Force-Enhanced Egocentric Learning), the first large-scale dataset pairing force measurements gathered from custom piezoresistive gloves with egocentric video. O…
Monocular Reconstruction of Neural Tactile Fields
Pavan Mantripragada, Siddhanth Deshmukh, Eadom Dessalene +2
Robots operating in the real world must plan through environments that deform, yield, and reconfigure under contact, requiring interaction-aware 3D representations that extend beyo…
Artificial Microsaccade Compensation: Stable Vision for an Ornithopter
Levi Burner, Guido de Croon, Yiannis Aloimonos
Animals with foveated vision, including humans, experience microsaccades, small, rapid eye movements that they are not aware of. Inspired by this phenomenon, we develop a method fo…
Odometry Without Correspondence from Inertially Constrained Ruled Surfaces
Chenqi Zhu, Levi Burner, Yiannis Aloimonos
Visual odometry techniques typically rely on feature extraction from a sequence of images and subsequent computation of optical flow. This point-to-point correspondence between two…
Adversarial Game-Theoretic Algorithm for Dexterous Grasp Synthesis
Yu Chen, Botao He, Yuemin Mao +7
For many complex tasks, multi-finger robot hands are poised to revolutionize how we interact with the world, but reliably grasping objects remains a significant challenge. We focus…
EREBUS: End-to-end Robust Event Based Underwater Simulation
Hitesh Kyatham, Arjun Suresh, Aadi Palnitkar +1
The underwater domain presents a vast array of challenges for roboticists and computer vision researchers alike, such as poor lighting conditions and high dynamic range scenes. In…
From Inpainting to Layer Decomposition: Repurposing Generative Inpainting Models for Image Layer Decomposition
Jingxi Chen, Yixiao Zhang, Xiaoye Qian +4
Images can be viewed as layered compositions, foreground objects over background, with potential occlusions. This layered representation enables independent editing of elements, of…
First Frame Is the Place to Go for Video Content Customization
Jingxi Chen, Zongxia Li, Zhichao Liu +6
What role does the first frame play in video generation models? Traditionally, it's viewed as the spatial-temporal starting point of a video, merely a seed for subsequent animation…
EmbodiSwap for Zero-Shot Robot Imitation Learning
Eadom Dessalene, Pavan Mantripragada, Michael Maynord +1
We introduce EmbodiSwap - a method for producing photorealistic synthetic robot overlays over human video. We employ EmbodiSwap for zero-shot imitation learning, bridging the embod…
NavMoE: Hybrid Model- and Learning-based Traversability Estimation for Local Navigation via Mixture of Experts
Botao He, Amir Hossein Shahidzadeh, Yu Chen +8
This paper explores traversability estimation for robot navigation. A key bottleneck in traversability estimation lies in efficiently achieving reliable and robust predictions whil…
Object Pose Estimation through Dexterous Touch
Amir-Hossein Shahidzadeh, Jiyue Zhu, Kezhou Chen +4
Robust object pose estimation is essential for manipulation and interaction tasks in robotics, particularly in scenarios where visual data is limited or sensitive to lighting, occl…
DREAM: Domain-aware Reasoning for Efficient Autonomous Underwater Monitoring
Zhenqi Wu, Abhinav Modi, Angelos Mavrogiannis +6
The ocean is warming and acidifying, increasing the risk of mass mortality events for temperature-sensitive shellfish such as oysters. This motivates the development of long-term m…
HiCL: Hippocampal-Inspired Continual Learning
Kushal Kapoor, Wyatt Mackey, Yiannis Aloimonos +1
We propose HiCL, a novel hippocampal-inspired dual-memory continual learning architecture designed to mitigate catastrophic forgetting by using elements inspired by the hippocampal…
ControlTac: Scaling Tactile Data with Physically Controlled Tactile Image Generation
Dongyu Luo, Kelin Yu, Amir-Hossein Shahidzadeh +3
Vision-based tactile sensing is widely used in perception, reconstruction, and robotic manipulation, yet collecting large-scale tactile data remains costly due to diverse sensor-ob…
NatSGLD: A Dataset with Speech, Gesture, Logic, and Demonstration for Robot Learning in Natural Human-Robot Interaction
Snehesh Shrestha, Yantian Zha, Saketh Banagiri +3
Recent advances in multimodal Human-Robot Interaction (HRI) datasets emphasize the integration of speech and gestures, allowing robots to absorb explicit knowledge and tacit unders…