794 citations · 2k across the 37 of their papers we have counts for
38 papers
GTAM: Geometry Grounded Track Anything Model
Chenming Zhu, Peizhou Cao, Jingli Lin +5
Human spatial understanding arises from jointly perceiving geometry and semantics, enabling consistent object identification and localization across viewpoints and time. Current vi…
MMSI-Video-Bench: A Holistic Benchmark for Video-Based Spatial Intelligence
Jingli Lin, Runsen Xu, Shaohao Zhu +11
Spatial understanding over continuous visual input is crucial for MLLMs to evolve into general-purpose assistants in physical environments. Yet there is still no comprehensive benc…
LabUtopia: High-Fidelity Simulation and Hierarchical Benchmark for Scientific Embodied Agents
Rui Li, Zixuan Hu, Wenxi Qu +13
Scientific embodied agents play a crucial role in modern laboratories by automating complex experimental workflows. Compared to typical household environments, laboratory settings…
Towards Latency-Aware 3D Streaming Perception for Autonomous Driving
Jiaqi Peng, Tai Wang, Jiangmiao Pang +1
Although existing 3D perception algorithms have demonstrated significant improvements in performance, their deployment on edge devices continues to encounter critical challenges du…
LLaVA-3D: A Simple yet Effective Pathway to Empowering LMMs with 3D-awareness
Chenming Zhu, Tai Wang, Wenwei Zhang +2
Recent advancements in Large Multimodal Models (LMMs) have greatly enhanced their proficiency in 2D visual understanding tasks, enabling them to effectively process and understand…
Multi-Object Tracking by Hierarchical Visual Representations
Jinkun Cao, Jiangmiao Pang, Kris Kitani
We propose a new visual hierarchical representation paradigm for multi-object tracking. It is more effective to discriminate between objects by attending to objects' compositional…
MGF: Mixed Gaussian Flow for Diverse Trajectory Prediction
Jiahe Chen, Jinkun Cao, Dahua Lin +2
To predict future trajectories, the normalizing flow with a standard Gaussian prior suffers from weak diversity. The ineffectiveness comes from the conflict between the fact of asy…
EmbodiedScan: A Holistic Multi-Modal 3D Perception Suite Towards Embodied AI
Tai Wang, Xiaohan Mao, Chenming Zhu +11
In the realm of computer vision and robotics, embodied agents are expected to explore their environment and carry out human instructions. This necessitates the ability to fully und…
OV-PARTS: Towards Open-Vocabulary Part Segmentation
Meng Wei, Xiaoyu Yue, Wenwei Zhang +3
Segmenting and recognizing diverse object parts is a crucial ability in applications spanning various computer vision and robotic tasks. While significant progress has been made in…
Understanding Masked Autoencoders From a Local Contrastive Perspective
Xiaoyu Yue, Lei Bai, Meng Wei +4
Masked AutoEncoder (MAE) has revolutionized the field of self-supervised learning with its simple yet effective masking and reconstruction strategies. However, despite achieving st…
Unified Human-Scene Interaction via Prompted Chain-of-Contacts
Zeqi Xiao, Tai Wang, Jingbo Wang +5
Human-Scene Interaction (HSI) is a vital component of fields like embodied AI and virtual reality. Despite advancements in motion quality and physical plausibility, two pivotal fac…
PointLLM: Empowering Large Language Models to Understand Point Clouds
Runsen Xu, Xiaolong Wang, Tai Wang +3
The unprecedented advancements in Large Language Models (LLMs) have shown a profound impact on natural language processing but are yet to fully embrace the realm of 3D understandin…
DORT: Modeling Dynamic Objects in Recurrent for Multi-Camera 3D Object Detection and Tracking
Qing Lian, Tai Wang, Dahua Lin +1
Recent multi-camera 3D object detectors usually leverage temporal information to construct multi-view stereo that alleviates the ill-posed depth estimation. However, they typically…
Transformer-Based Visual Segmentation: A Survey
Xiangtai Li, Henghui Ding, Haobo Yuan +6
Visual segmentation seeks to partition images, video frames, or point clouds into multiple segments or groups. This technique has numerous real-world applications, such as autonomo…
MV-JAR: Masked Voxel Jigsaw and Reconstruction for LiDAR-Based Self-Supervised Pre-Training
Runsen Xu, Tai Wang, Wenwei Zhang +4
This paper introduces the Masked Voxel Jigsaw and Reconstruction (MV-JAR) method for LiDAR-based self-supervised pre-training and a carefully designed data-efficient 3D object dete…
Position-Guided Point Cloud Panoptic Segmentation Transformer
Zeqi Xiao, Wenwei Zhang, Tai Wang +3
DEtection TRansformer (DETR) started a trend that uses a group of learnable queries for unified visual perception. This work begins by applying this appealing paradigm to LiDAR-bas…
Tube-Link: A Flexible Cross Tube Framework for Universal Video Segmentation
Xiangtai Li, Haobo Yuan, Wenwei Zhang +3
Video segmentation aims to segment and track every pixel in diverse scenarios accurately. In this paper, we present Tube-Link, a versatile framework that addresses multiple core ta…
Dense Distinct Query for End-to-End Object Detection
Shilong Zhang, Xinjiang Wang, Jiaqi Wang +5
One-to-one label assignment in object detection has successfully obviated the need for non-maximum suppression (NMS) as postprocessing and makes the pipeline end-to-end. However, i…
QDTrack: Quasi-Dense Similarity Learning for Appearance-Only Multiple Object Tracking
Tobias Fischer, Thomas E. Huang, Jiangmiao Pang +4
Similarity learning has been recognized as a crucial step for object tracking. However, existing multiple object tracking methods only use sparse ground truth matching as the train…
Monocular 3D Object Detection with Depth from Motion
Tai Wang, Jiangmiao Pang, Dahua Lin
Perceiving 3D objects from monocular inputs is crucial for robotic systems, given its economy compared to multi-sensor settings. It is notably difficult as a single image can not p…
What Are Expected Queries in End-to-End Object Detection?
Shilong Zhang, Xinjiang Wang, Jiaqi Wang +2
End-to-end object detection is rapidly progressed after the emergence of DETR. DETRs use a set of sparse queries that replace the dense candidate boxes in most traditional detector…
Video K-Net: A Simple, Strong, and Unified Baseline for Video Segmentation
Xiangtai Li, Wenwei Zhang, Jiangmiao Pang +4
This paper presents Video K-Net, a simple, strong, and unified framework for fully end-to-end video panoptic segmentation. The method is built upon K-Net, a method that unifies ima…
Observation-Centric SORT: Rethinking SORT for Robust Multi-Object Tracking
Jinkun Cao, Jiangmiao Pang, Xinshuo Weng +2
Kalman filter (KF) based methods for multi-object tracking (MOT) make an assumption that objects move linearly. While this assumption is acceptable for very short periods of occlus…
Dense Siamese Network for Dense Unsupervised Learning
Wenwei Zhang, Jiangmiao Pang, Kai Chen +1
This paper presents Dense Siamese Network (DenseSiam), a simple unsupervised learning framework for dense prediction tasks. It learns visual representations by maximizing the simil…