1.4k citations · 4.9k across the 44 of their papers we have counts for
57 papers
ViPE: Video Pose Engine for 3D Geometric Perception
Jiahui Huang, Qunjie Zhou, Hesam Rabeti +12
Accurate 3D geometric perception is an important prerequisite for a wide range of spatial AI systems. While state-of-the-art methods depend on large-scale training data, acquiring…
Cosmos-Drive-Dreams: Scalable Synthetic Driving Data Generation with World Foundation Models
Xuanchi Ren, Yifan Lu, Tianshi Cao +13
Collecting and annotating real-world data for safety-critical physical AI systems, such as Autonomous Vehicle (AV), is time-consuming and costly. It is especially challenging to ca…
Native Segmentation Vision Transformers
Guillem Brasó, Aljoša Ošep, Laura Leal-Taixé
Uniform downsampling remains the de facto standard for reducing spatial resolution in vision backbones. In this work, we propose an alternative design built around a content-aware…
Cosmos World Foundation Model Platform for Physical AI
NVIDIA, :, Niket Agarwal +76
Physical AI needs to be trained digitally first. It needs a digital twin of itself, the policy model, and a digital twin of the world, the world model. In this paper, we present th…
Lidar Panoptic Segmentation in an Open World
Anirudh S Chakravarthy, Meghana Reddy Ganesina, Peiyun Hu +4
Addressing Lidar Panoptic Segmentation (LPS ) is crucial for safe deployment of autonomous vehicles. LPS aims to recognize and segment lidar points w.r.t. a pre-defined vocabulary…
NOVIS: A Case for End-to-End Near-Online Video Instance Segmentation
Tim Meinhardt, Matt Feiszli, Yuchen Fan +2
Until recently, the Video Instance Segmentation (VIS) community operated under the common belief that offline methods are generally superior to a frame by frame online processing.…
HOLISMOKES -- XI. Evaluation of supervised neural networks for strong-lens searches in ground-based imaging surveys
R. Canameras, S. Schuldt, Y. Shu +10
While supervised neural networks have become state of the art for identifying the rare strong gravitational lenses from large imaging data sets, their selection remains significant…
Learning to Discover and Detect Objects
Vladimir Fomenko, Ismail Elezi, Deva Ramanan +2
We tackle the problem of novel class discovery and localization (NCDL). In this setting, we assume a source dataset with supervision for only some object classes. Instances of othe…
Multi-Object Tracking and Segmentation via Neural Message Passing
Guillem Braso, Orcun Cetintas, Laura Leal-Taixe
Graphs offer a natural way to formulate Multiple Object Tracking (MOT) and Multiple Object Tracking and Segmentation (MOTS) within the tracking-by-detection paradigm. However, they…
Simple Cues Lead to a Strong Multi-Object Tracker
Jenny Seidenschwarz, Guillem Brasó, Victor Castro Serrano +2
For a long time, the most common paradigm in Multi-Object Tracking was tracking-by-detection (TbD), where objects are first detected and then associated over video frames. For asso…
Text2Pos: Text-to-Point-Cloud Cross-Modal Localization
Manuel Kolmet, Qunjie Zhou, Aljosa Osep +1
Natural language-based communication with mobile devices and home appliances is becoming increasingly popular and has the potential to become natural for communicating with mobile…
The Group Loss++: A deeper look into group loss for deep metric learning
Ismail Elezi, Jenny Seidenschwarz, Laurin Wagner +4
Deep metric learning has yielded impressive results in tasks such as clustering and image retrieval by leveraging neural networks to obtain highly discriminative feature embeddings…
Is Geometry Enough for Matching in Visual Localization?
Qunjie Zhou, Sérgio Agostinho, Aljosa Osep +1
In this paper, we propose to go beyond the well-established approach to vision-based localization that relies on visual descriptor matching between a query image and a 3D point clo…
MOTSynth: How Can Synthetic Data Help Pedestrian Detection and Tracking?
Matteo Fabbri, Guillem Braso, Gianluca Maugeri +6
Deep learning-based methods for video pedestrian detection and tracking require large volumes of training data to achieve good performance. However, data acquisition in crowded pub…
MG-GAN: A Multi-Generator Model Preventing Out-of-Distribution Samples in Pedestrian Trajectory Prediction
Patrick Dendorfer, Sven Elflein, Laura Leal-Taixé
Pedestrian trajectory prediction is challenging due to its uncertain and multimodal nature. While generative adversarial networks can learn a distribution over future trajectories,…
(Just) A Spoonful of Refinements Helps the Registration Error Go Down
Sérgio Agostinho, Aljoša Ošep, Alessio Del Bue +1
We tackle data-driven 3D point cloud registration. Given point correspondences, the standard Kabsch algorithm provides an optimal rotation estimate. This allows to train registrati…
DeepLab2: A TensorFlow Library for Deep Labeling
Mark Weber, Huiyu Wang, Siyuan Qiao +12
DeepLab2 is a TensorFlow library for deep labeling, aiming to provide a state-of-the-art and easy-to-use TensorFlow codebase for general dense pixel prediction problems in computer…
Not All Labels Are Equal: Rationalizing The Labeling Costs for Training Object Detection
Ismail Elezi, Zhiding Yu, Anima Anandkumar +2
Deep neural networks have reached high accuracy on object detection but their success hinges on large amounts of labeled data. To reduce the labels dependency, various active learn…
The 2021 Image Similarity Dataset and Challenge
Matthijs Douze, Giorgos Tolias, Ed Pizzi +9
This paper introduces a new benchmark for large-scale image similarity detection. This benchmark is used for the Image Similarity Challenge at NeurIPS'21 (ISC2021). The goal is to…
EagerMOT: 3D Multi-Object Tracking via Sensor Fusion
Aleksandr Kim, Aljoša Ošep, Laura Leal-Taixé
Multi-object tracking (MOT) enables mobile robots to perform well-informed motion planning and navigation by localizing surrounding objects in 3D space and time. Existing methods r…
Opening up Open-World Tracking
Yang Liu, Idil Esen Zulfikar, Jonathon Luiten +5
Tracking and detecting any object, including ones never-seen-before during model training, is a crucial but elusive capability of autonomous systems. An autonomous agent that is bl…
Vision-Based Mobile Robotics Obstacle Avoidance With Deep Reinforcement Learning
Patrick Wenzel, Torsten Schön, Laura Leal-Taixé +1
Obstacle avoidance is a fundamental and challenging problem for autonomous navigation of mobile robots. In this paper, we consider the problem of obstacle avoidance in simple 3D en…
4D Panoptic LiDAR Segmentation
Mehmet Aygün, Aljoša Ošep, Mark Weber +4
Temporal semantic scene understanding is critical for self-driving cars or robots operating in dynamic environments. In this paper, we propose 4D panoptic LiDAR segmentation to ass…
STEP: Segmenting and Tracking Every Pixel
Mark Weber, Jun Xie, Maxwell Collins +10
The task of assigning semantic classes and track identities to every pixel in a video is called video panoptic segmentation. Our work is the first that targets this task in a real-…