37 citations · 101 across the 38 of their papers we have counts for
41 papers
HapticWorld: an Interactive World Simulator with Real-time Torque Feedback
Shaoting Peng, Litian Liang, Yixuan Wang +4
Contact-rich manipulation depends on force sensing that is hard to infer from visual signals alone, both for collecting demonstrations and for training policies. Force-annotated da…
ARGUS: Aligning Robot Scene Geometry Under Shifting Views with Large 3D Vision Models
Rishik Sathua, Haonan Chen, Katherine Driggs-Campbell
Large-scale visuomotor policies have demonstrated impressive performance across a wide range of robot manipulation tasks. However, despite this success, manipulation polices often…
Brick-Composer: Using MLLMs for Assembly with Diverse Bricks
Jiateng Liu, Bingxuan Li, Zhenhailong Wang +8
We dream of AI agents that can read arbitrary designs and construct real-world objects from reusable building blocks. As a first step toward this vision, we study whether multimoda…
DiscreteRTC: Discrete Diffusion Policies are Natural Asynchronous Executors
Pengcheng Wang, Kaiwen Hong, Chensheng Peng +4
Unlike chatbots, physical AI must act while the world keeps evolving. Therefore, the inter-chunk pause of synchronous executors are fatal for dynamic tasks regardless of how fast t…
Tactile-Based Human Intent Recognition for Robot Assistive Navigation
Shaoting Peng, Dakarai Crowder, Wenzhen Yuan +1
Robot assistive navigation (RAN) is critical for enhancing the mobility and independence of the growing population of mobility-impaired individuals. However, existing systems often…
Trust-Aware Embodied Bayesian Persuasion for Mixed-Autonomy
Shaoting Peng, Katherine Driggs-Campbell, Roy Dong
Safe and efficient interaction between autonomous vehicles (AVs) and human-driven vehicles (HVs) is a critical challenge for future transportation systems. While game-theoretic mod…
Multi-Modal Manipulation via Multi-Modal Policy Consensus
Haonan Chen, Jiaming Xu, Hongyu Chen +7
Effectively integrating diverse sensory modalities is crucial for robotic manipulation. However, the typical approach of feature concatenation is often suboptimal: dominant modalit…
Characterizing the Robustness of Black-Box LLM Planners Under Perturbed Observations with Adaptive Stress Testing
Neeloy Chakraborty, John Pohovey, Melkior Ornik +1
Large language models (LLMs) have recently demonstrated success in decision-making tasks including planning, control, and prediction, but their tendency to hallucinate unsafe and u…
Do You Know the Way? Human-in-the-Loop Understanding for Fast Traversability Estimation in Mobile Robotics
Andre Schreiber, Katherine Driggs-Campbell
The increasing use of robots in unstructured environments necessitates the development of effective perception and navigation strategies to enable field robots to successfully perf…
Tool-as-Interface: Learning Robot Policies from Observing Human Tool Use
Haonan Chen, Cheng Zhu, Shuijing Liu +2
Tool use is essential for enabling robots to perform complex real-world tasks, but learning such skills requires extensive datasets. While teleoperation is widely used, it is slow,…
Learning Coordinated Bimanual Manipulation Policies using State Diffusion and Inverse Dynamics Models
Haonan Chen, Jiaming Xu, Lily Sheng +4
When performing tasks like laundry, humans naturally coordinate both hands to manipulate objects and anticipate how their actions will change the state of the clothes. However, ach…
Towards Uncertainty Unification: A Case Study for Preference Learning
Shaoting Peng, Haonan Chen, Katherine Driggs-Campbell
Learning human preferences is essential for human-robot interaction, as it enables robots to adapt their behaviors to align with human expectations and goals. However, the inherent…
An Expert Ensemble for Detecting Anomalous Scenes, Interactions, and Behaviors in Autonomous Driving
Tianchen Ji, Neeloy Chakraborty, Andre Schreiber +1
As automated vehicles enter public roads, safety in a near-infinite number of driving scenarios becomes one of the major concerns for the widespread adoption of fully autonomous dr…
Interaction-aware Conformal Prediction for Crowd Navigation
Zhe Huang, Tianchen Ji, Heling Zhang +3
During crowd navigation, robot motion plan needs to consider human motion uncertainty, and the human motion uncertainty is dependent on the robot motion plan. We introduce Interact…
HEIGHT: Heterogeneous Interaction Graph Transformer for Robot Navigation in Crowded and Constrained Environments
Shuijing Liu, Haochen Xia, Fatemeh Cheraghi Pouria +5
We study the problem of robot navigation in dense and interactive crowds with static constraints such as corridors and furniture. Previous methods fail to consider all types of spa…
Towards Real-Time Generation of Delay-Compensated Video Feeds for Outdoor Mobile Robot Teleoperation
Neeloy Chakraborty, Yixiao Fang, Andre Schreiber +6
Teleoperation is an important technology to enable supervisors to control agricultural robots remotely. However, environmental factors in dense crop rows and limitations in network…
Topology-Guided ORCA: Smooth Multi-Agent Motion Planning in Constrained Environments
Fatemeh Cheraghi Pouria, Zhe Huang, Ananya Yammanuru +2
We present Topology-Guided ORCA as an alternative simulator to replace ORCA for planning smooth multi-agent motions in environments with static obstacles. Despite the impressive pe…
Human-Agent Joint Learning for Efficient Robot Manipulation Skill Acquisition
Shengcheng Luo, Quanquan Peng, Jun Lv +4
Employing a teleoperation system for gathering demonstrations offers the potential for more efficient learning of robot manipulation. However, teleoperating a robot arm equipped wi…
Cooperative Advisory Residual Policies for Congestion Mitigation
Aamir Hasan, Neeloy Chakraborty, Haonan Chen +3
Fleets of autonomous vehicles can mitigate traffic congestion through simple actions, thus improving many socioeconomic factors such as commute time and gas costs. However, these a…
Lessons in Cooperation: A Qualitative Analysis of Driver Sentiments towards Real-Time Advisory Systems from a Driving Simulator User Study
Aamir Hasan, Neeloy Chakraborty, Haonan Chen +2
Real-time Advisory (RTA) systems, such as navigational and eco-driving assistants, are becoming increasingly ubiquitous in vehicles due to their benefits for users and society. Unt…
W-RIZZ: A Weakly-Supervised Framework for Relative Traversability Estimation in Mobile Robotics
Andre Schreiber, Arun N. Sivakumar, Peter Du +3
Successful deployment of mobile robots in unstructured domains requires an understanding of the environment and terrain to avoid hazardous areas, getting stuck, and colliding with…
Structured Graph Network for Constrained Robot Crowd Navigation with Low Fidelity Simulation
Shuijing Liu, Kaiwen Hong, Neeloy Chakraborty +1
We investigate the feasibility of deploying reinforcement learning (RL) policies for constrained crowd navigation using a low-fidelity simulator. We introduce a representation of t…
Hallucination Detection in Foundation Models for Decision-Making: A Flexible Definition and Review of the State of the Art
Neeloy Chakraborty, Melkior Ornik, Katherine Driggs-Campbell
Autonomous systems are soon to be ubiquitous, spanning manufacturing, agriculture, healthcare, entertainment, and other industries. Most of these systems are developed with modular…
Beyond the Dashboard: Investigating Distracted Driver Communication Preferences for ADAS
Aamir Hasan, D. Livingston McPherson, Melissa Miles +1
Distracted driving is a major cause of road fatalities. With improvements in driver (in)attention detection, these distracted situations can be caught early to alert drivers and im…