137 citations · 258 across the 25 of their papers we have counts for
25 papers
Chain Of Interaction Benchmark (COIN): When Reasoning meets Embodied Interaction
Xianhao Wang, Xiaojian Ma, Haozhe Hu +7
Generalist embodied agents must perform interactive, causally-dependent reasoning, continually interacting with the environment, acquiring information, and updating plans to solve…
Integration of Robot and Scene Kinematics for Sequential Mobile Manipulation Planning
Ziyuan Jiao, Yida Niu, Zeyu Zhang +5
We present a Sequential Mobile Manipulation Planning (SMMP) framework that can solve long-horizon multi-step mobile manipulation tasks with coordinated whole-body motion, even when…
Embedding high-resolution touch across robotic hands enables adaptive human-like grasping
Zihang Zhao, Wanlin Li, Yuyang Li +9
Developing robotic hands that adapt to real-world dynamics remains a fundamental challenge in robotics and machine intelligence. Despite significant advances in replicating human h…
M2Diffuser: Diffusion-based Trajectory Optimization for Mobile Manipulation in 3D Scenes
Sixu Yan, Zeyu Zhang, Muzhi Han +7
Recent advances in diffusion models have opened new avenues for research into embodied AI agents and robotics. Despite significant achievements in complex robotic locomotion and sk…
MiniTac: An Ultra-Compact 8 mm Vision-Based Tactile Sensor for Enhanced Palpation in Robot-Assisted Minimally Invasive Surgery
Wanlin Li, Zihang Zhao, Leiyao Cui +4
Robot-assisted minimally invasive surgery (RAMIS) provides substantial benefits over traditional open and laparoscopic methods. However, a significant limitation of RAMIS is the su…
M3Bench: Benchmarking Whole-body Motion Generation for Mobile Manipulation in 3D Scenes
Zeyu Zhang, Sixu Yan, Muzhi Han +4
We propose M3Bench, a new benchmark for whole-body motion generation in mobile manipulation tasks. Given a 3D scene context, M3Bench requires an embodied agent to reason about its…
LLM3:Large Language Model-based Task and Motion Planning with Motion Failure Reasoning
Shu Wang, Muzhi Han, Ziyuan Jiao +4
Conventional Task and Motion Planning (TAMP) approaches rely on manually crafted interfaces connecting symbolic task planning with continuous motion generation. These domain-specif…
On the Emergence of Symmetrical Reality
Zhenliang Zhang, Zeyu Zhang, Ziyuan Jiao +4
Artificial intelligence (AI) has revolutionized human cognitive abilities and facilitated the development of new AI entities capable of interacting with humans in both physical and…
Aggregating Single-wheeled Mobile Robots for Omnidirectional Movements
Meng Wang, Yao Su, Hang Li +3
This paper presents a novel modular robot system that can self-reconfigure to achieve omnidirectional movements for collaborative object transportation. Each robotic module is equi…
Part-level Scene Reconstruction Affords Robot Interaction
Zeyu Zhang, Lexing Zhang, Zaijin Wang +5
Existing methods for reconstructing interactive scenes primarily focus on replacing reconstructed objects with CAD models retrieved from a limited database, resulting in significan…
Sequential Manipulation Planning for Over-actuated Unmanned Aerial Manipulators
Yao Su, Jiarui Li, Ziyuan Jiao +5
We investigate the sequential manipulation planning problem for unmanned aerial manipulators (UAMs). Unlike prior work that primarily focuses on one-step manipulation tasks, sequen…
Probabilistic Visibility-Aware Trajectory Planning for Target Tracking in Cluttered Environments
Han Gao, Pengying Wu, Yao Su +4
Target tracking has numerous significant civilian and military applications, and maintaining the visibility of the target plays a vital role in ensuring the success of the tracking…
Rearrange Indoor Scenes for Human-Robot Co-Activity
Weiqi Wang, Zihang Zhao, Ziyuan Jiao +3
We present an optimization-based framework for rearranging indoor furniture to accommodate human-robot co-activities better. The rearrangement aims to afford sufficient accessible…
Downwash-aware Control Allocation for Over-actuated UAV Platforms
Yao Su, Chi Chu, Meng Wang +4
Tracking position and orientation independently affords more agile maneuver for over-actuated multirotor Unmanned Aerial Vehicles (UAVs) while introducing undesired downwash effect…
Sequential Manipulation Planning on Scene Graph
Ziyuan Jiao, Yida Niu, Zeyu Zhang +3
We devise a 3D scene graph representation, contact graph+ (cg+), for efficient sequential task planning. Augmented with predicate-like attributes, this contact graph-based represen…
Understanding Physical Effects for Effective Tool-use
Zeyu Zhang, Ziyuan Jiao, Weiqi Wang +3
We present a robot learning and planning framework that produces an effective tool-use strategy with the least joint efforts, capable of handling objects different from training. L…
Object Gathering with a Tethered Robot Duo
Yao Su, Yuhong Jiang, Yixin Zhu +1
We devise a cooperative planning framework to generate optimal trajectories for a tethered robot duo, who is tasked to gather scattered objects spread in a large area using a flexi…
Consolidating Kinematic Models to Promote Coordinated Mobile Manipulations
Ziyuan Jiao, Zeyu Zhang, Xin Jiang +4
We construct a Virtual Kinematic Chain (VKC) that readily consolidates the kinematics of the mobile base, the arm, and the object to be manipulated in mobile manipulations. Accordi…
Efficient Task Planning for Mobile Manipulation: a Virtual Kinematic Chain Perspective
Ziyuan Jiao, Zeyu Zhang, Weiqi Wang +4
We present a Virtual Kinematic Chain (VKC) perspective, a simple yet effective method, to improve task planning efficacy for mobile manipulation. By consolidating the kinematics of…
Reconstructing Interactive 3D Scenes by Panoptic Mapping and CAD Model Alignments
Muzhi Han, Zeyu Zhang, Ziyuan Jiao +4
In this paper, we rethink the problem of scene reconstruction from an embodied agent's perspective: While the classic view focuses on the reconstruction accuracy, our new perspecti…
Human-Robot Interaction in a Shared Augmented Reality Workspace
Shuwen Qiu, Hangxin Liu, Zeyu Zhang +2
We design and develop a new shared Augmented Reality (AR) workspace for Human-Robot Interaction (HRI), which establishes a bi-directional communication between human agents and rob…
Joint Inference of States, Robot Knowledge, and Human (False-)Beliefs
Tao Yuan, Hangxin Liu, Lifeng Fan +4
Aiming to understand how human (false-)belief--a core socio-cognitive ability--would affect human interactions with robots, this paper proposes to adopt a graphical model to unify…
Congestion-aware Evacuation Routing using Augmented Reality Devices
Zeyu Zhang, Hangxin Liu, Ziyuan Jiao +2
We present a congestion-aware routing solution for indoor evacuation, which produces real-time individual-customized evacuation routes among multiple destinations while keeping tra…
Dark, Beyond Deep: A Paradigm Shift to Cognitive AI with Humanlike Common Sense
Yixin Zhu, Tao Gao, Lifeng Fan +9
Recent progress in deep learning is essentially based on a "big data for small tasks" paradigm, under which massive amounts of data are used to train a classifier for a single narr…