137 citations · 418 across the 29 of their papers we have counts for
29 papers
Part-level Scene Reconstruction Affords Robot Interaction
Zeyu Zhang, Lexing Zhang, Zaijin Wang +5
Existing methods for reconstructing interactive scenes primarily focus on replacing reconstructed objects with CAD models retrieved from a limited database, resulting in significan…
Sequential Manipulation Planning for Over-actuated Unmanned Aerial Manipulators
Yao Su, Jiarui Li, Ziyuan Jiao +5
We investigate the sequential manipulation planning problem for unmanned aerial manipulators (UAMs). Unlike prior work that primarily focuses on one-step manipulation tasks, sequen…
STRAP: Structured Object Affordance Segmentation with Point Supervision
Leiyao Cui, Xiaoxue Chen, Hao Zhao +2
With significant annotation savings, point supervision has been proven effective for numerous 2D and 3D scene understanding problems. This success is primarily attributed to the st…
Rearrange Indoor Scenes for Human-Robot Co-Activity
Weiqi Wang, Zihang Zhao, Ziyuan Jiao +3
We present an optimization-based framework for rearranging indoor furniture to accommodate human-robot co-activities better. The rearrangement aims to afford sufficient accessible…
Diffusion-based Generation, Optimization, and Planning in 3D Scenes
Siyuan Huang, Zan Wang, Puhao Li +5
We introduce SceneDiffuser, a conditional generative model for 3D scene understanding. SceneDiffuser provides a unified model for solving scene-conditioned generation, optimization…
Full-Body Articulated Human-Object Interaction
Nan Jiang, Tengyu Liu, Zhexuan Cao +6
Fine-grained capturing of 3D HOI boosts human activity understanding and facilitates downstream visual tasks, including action recognition, holistic scene reconstruction, and human…
HUMANISE: Language-conditioned Human Motion Generation in 3D Scenes
Zan Wang, Yixin Chen, Tengyu Liu +3
Learning to generate diverse scene-aware and goal-oriented human motions in 3D scenes remains challenging due to the mediocre characteristics of the existing datasets on Human-Scen…
Downwash-aware Control Allocation for Over-actuated UAV Platforms
Yao Su, Chi Chu, Meng Wang +4
Tracking position and orientation independently affords more agile maneuver for over-actuated multirotor Unmanned Aerial Vehicles (UAVs) while introducing undesired downwash effect…
Sequential Manipulation Planning on Scene Graph
Ziyuan Jiao, Yida Niu, Zeyu Zhang +3
We devise a 3D scene graph representation, contact graph+ (cg+), for efficient sequential task planning. Augmented with predicate-like attributes, this contact graph-based represen…
Understanding Physical Effects for Effective Tool-use
Zeyu Zhang, Ziyuan Jiao, Weiqi Wang +3
We present a robot learning and planning framework that produces an effective tool-use strategy with the least joint efforts, capable of handling objects different from training. L…
Object Gathering with a Tethered Robot Duo
Yao Su, Yuhong Jiang, Yixin Zhu +1
We devise a cooperative planning framework to generate optimal trajectories for a tethered robot duo, who is tasked to gather scattered objects spread in a large area using a flexi…
HALMA: Humanlike Abstraction Learning Meets Affordance in Rapid Problem Solving
Sirui Xie, Xiaojian Ma, Peiyu Yu +3
Humans learn compositional and causal abstraction, \ie, knowledge, in response to the structure of naturalistic tasks. When presented with a problem-solving task involving some obj…
LEMMA: A Multi-view Dataset for Learning Multi-agent Multi-task Activities
Baoxiong Jia, Yixin Chen, Siyuan Huang +2
Understanding and interpreting human actions is a long-standing challenge and a critical indicator of perception in artificial intelligence. However, a few imperative components of…
Human-Robot Interaction in a Shared Augmented Reality Workspace
Shuwen Qiu, Hangxin Liu, Zeyu Zhang +2
We design and develop a new shared Augmented Reality (AR) workspace for Human-Robot Interaction (HRI), which establishes a bi-directional communication between human agents and rob…
Joint Inference of States, Robot Knowledge, and Human (False-)Beliefs
Tao Yuan, Hangxin Liu, Lifeng Fan +4
Aiming to understand how human (false-)belief--a core socio-cognitive ability--would affect human interactions with robots, this paper proposes to adopt a graphical model to unify…
Congestion-aware Evacuation Routing using Augmented Reality Devices
Zeyu Zhang, Hangxin Liu, Ziyuan Jiao +2
We present a congestion-aware routing solution for indoor evacuation, which produces real-time individual-customized evacuation routes among multiple destinations while keeping tra…
Machine Number Sense: A Dataset of Visual Arithmetic Problems for Abstract and Relational Reasoning
Wenhe Zhang, Chi Zhang, Yixin Zhu +1
As a comprehensive indicator of mathematical thinking and intelligence, the number sense (Dehaene 2011) bridges the induction of symbolic concepts and the competence of problem-sol…
Dark, Beyond Deep: A Paradigm Shift to Cognitive AI with Humanlike Common Sense
Yixin Zhu, Tao Gao, Lifeng Fan +9
Recent progress in deep learning is essentially based on a "big data for small tasks" paradigm, under which massive amounts of data are used to train a classifier for a single narr…
PerspectiveNet: 3D Object Detection from a Single RGB Image via Perspective Points
Siyuan Huang, Yixin Chen, Tao Yuan +3
Detecting 3D objects from a single RGB image is intrinsically ambiguous, thus requiring appropriate prior knowledge and intermediate representations as constraints to reduce the un…
Learning Perceptual Inference by Contrasting
Chi Zhang, Baoxiong Jia, Feng Gao +3
"Thinking in pictures," [1] i.e., spatial-temporal reasoning, effortless and instantaneous for humans, is believed to be a significant ability to perform logical induction and a cr…
Theory-based Causal Transfer: Integrating Instance-level Induction and Abstract-level Structure Learning
Mark Edmonds, Xiaojian Ma, Siyuan Qi +3
Learning transferable knowledge across similar but different settings is a fundamental component of generalized intelligence. In this paper, we approach the transfer learning chall…
Holistic++ Scene Understanding: Single-view 3D Holistic Scene Parsing and Human Pose Estimation with Human-Object Interaction and Physical Commonsense
Yixin Chen, Siyuan Huang, Tao Yuan +3
We propose a new 3D holistic++ scene understanding problem, which jointly tackles two tasks from a single-view image: (i) holistic scene parsing and reconstruction---3D estimations…
VRGym: A Virtual Testbed for Physical and Interactive AI
Xu Xie, Hangxin Liu, Zhenliang Zhang +5
We propose VRGym, a virtual reality testbed for realistic human-robot interaction. Different from existing toolkits and virtual reality environments, the VRGym emphasizes on buildi…
RAVEN: A Dataset for Relational and Analogical Visual rEasoNing
Chi Zhang, Feng Gao, Baoxiong Jia +2
Dramatic progress has been witnessed in basic vision tasks involving low-level perception, such as object recognition, detection, and tracking. Unfortunately, there is still an eno…