1.1k citations · 1.4k across the 24 of their papers we have counts for
34 papers
Foundation4D: Towards a unified perception and reconstruction model for autonomous vehicles
NVIDIA, :, Sérgio Agostinho +25
We introduce Foundation4D, a unified feed-forward model for geometry estimation, 3D detection, semantic occupancy, and Gaussian reconstruction from multi-camera driving video. A sh…
Addressable Memory for Video World Models
Xindi Wu, Sven Elflein, James Lucas +5
We study visual persistence in interactive video world models. These models rely on a Key-Value (KV) cache as a growing visual memory to carry forward previously generated frames.…
VGG-T: Offline Feed-Forward 3D Reconstruction at Scale
Sven Elflein, Ruilong Li, Sérgio Agostinho +4
We present a scalable 3D reconstruction model that addresses a critical limitation in offline feed-forward methods: their computational and memory requirements grow quadratically w…
SVAG-Bench: A Large-Scale Benchmark for Multi-Instance Spatio-temporal Video Action Grounding
Tanveer Hannan, Shuaicong Wu, Mark Weber +6
A truly capable AI system must do more than detect objects or recognize activities in isolation. It must form unified, grounded representations of who is acting, what they are doin…
Native Segmentation Vision Transformers
Guillem Brasó, Aljoša Ošep, Laura Leal-Taixé
Uniform downsampling remains the de facto standard for reducing spatial resolution in vision backbones. In this work, we propose an alternative design built around a content-aware…
Zero-Shot 4D Lidar Panoptic Segmentation
Yushan Zhang, Aljoša Ošep, Laura Leal-Taixé +1
Zero-shot 4D segmentation and recognition of arbitrary objects in Lidar is crucial for embodied navigation, with applications ranging from streaming perception to semantic mapping…
Lidar Panoptic Segmentation in an Open World
Anirudh S Chakravarthy, Meghana Reddy Ganesina, Peiyun Hu +4
Addressing Lidar Panoptic Segmentation (LPS ) is crucial for safe deployment of autonomous vehicles. LPS aims to recognize and segment lidar points w.r.t. a pre-defined vocabulary…
SeMoLi: What Moves Together Belongs Together
Jenny Seidenschwarz, Aljoša Ošep, Francesco Ferroni +2
We tackle semi-supervised object detection based on motion cues. Recent results suggest that heuristic-based clustering methods in conjunction with object trackers can be used to p…
Pix2Map: Cross-modal Retrieval for Inferring Street Maps from Images
Xindi Wu, KwunFung Lau, Francesco Ferroni +2
Self-driving vehicles rely on urban street maps for autonomous navigation. In this paper, we introduce Pix2Map, a method for inferring urban street map topology directly from ego-v…
Learning to Discover and Detect Objects
Vladimir Fomenko, Ismail Elezi, Deva Ramanan +2
We tackle the problem of novel class discovery and localization (NCDL). In this setting, we assume a source dataset with supervision for only some object classes. Instances of othe…
Quo Vadis: Is Trajectory Forecasting the Key Towards Long-Term Multi-Object Tracking?
Patrick Dendorfer, Vladimir Yugay, Aljoša Ošep +1
Recent developments in monocular multi-object tracking have been very successful in tracking visible objects and bridging short occlusion gaps, mainly relying on data-driven appear…
PolarMOT: How Far Can Geometric Relations Take Us in 3D Multi-Object Tracking?
Aleksandr Kim, Guillem Brasó, Aljoša Ošep +1
Most (3D) multi-object tracking methods rely on appearance-based cues for data association. By contrast, we investigate how far we can get by only encoding geometric relationships…
Text2Pos: Text-to-Point-Cloud Cross-Modal Localization
Manuel Kolmet, Qunjie Zhou, Aljosa Osep +1
Natural language-based communication with mobile devices and home appliances is becoming increasingly popular and has the potential to become natural for communicating with mobile…
Is Geometry Enough for Matching in Visual Localization?
Qunjie Zhou, Sérgio Agostinho, Aljosa Osep +1
In this paper, we propose to go beyond the well-established approach to vision-based localization that relies on visual descriptor matching between a query image and a 3D point clo…
MOTSynth: How Can Synthetic Data Help Pedestrian Detection and Tracking?
Matteo Fabbri, Guillem Braso, Gianluca Maugeri +6
Deep learning-based methods for video pedestrian detection and tracking require large volumes of training data to achieve good performance. However, data acquisition in crowded pub…
(Just) A Spoonful of Refinements Helps the Registration Error Go Down
Sérgio Agostinho, Aljoša Ošep, Alessio Del Bue +1
We tackle data-driven 3D point cloud registration. Given point correspondences, the standard Kabsch algorithm provides an optimal rotation estimate. This allows to train registrati…
EagerMOT: 3D Multi-Object Tracking via Sensor Fusion
Aleksandr Kim, Aljoša Ošep, Laura Leal-Taixé
Multi-object tracking (MOT) enables mobile robots to perform well-informed motion planning and navigation by localizing surrounding objects in 3D space and time. Existing methods r…
Opening up Open-World Tracking
Yang Liu, Idil Esen Zulfikar, Jonathon Luiten +5
Tracking and detecting any object, including ones never-seen-before during model training, is a crucial but elusive capability of autonomous systems. An autonomous agent that is bl…
4D Panoptic LiDAR Segmentation
Mehmet Aygün, Aljoša Ošep, Mark Weber +4
Temporal semantic scene understanding is critical for self-driving cars or robots operating in dynamic environments. In this paper, we propose 4D panoptic LiDAR segmentation to ass…
STEP: Segmenting and Tracking Every Pixel
Mark Weber, Jun Xie, Maxwell Collins +10
The task of assigning semantic classes and track identities to every pixel in a video is called video panoptic segmentation. Our work is the first that targets this task in a real-…
Goal-GAN: Multimodal Trajectory Prediction Based on Goal Position Estimation
Patrick Dendorfer, Aljoša Ošep, Laura Leal-Taixé
In this paper, we present Goal-GAN, an interpretable and end-to-end trainable model for human trajectory prediction. Inspired by human navigation, we model the task of trajectory p…
MOTChallenge: A Benchmark for Single-Camera Multiple Target Tracking
Patrick Dendorfer, Aljoša Ošep, Anton Milan +5
Standardized benchmarks have been crucial in pushing the performance of computer vision algorithms, especially since the advent of deep learning. Although leaderboards should not b…
HOTA: A Higher Order Metric for Evaluating Multi-Object Tracking
Jonathon Luiten, Aljosa Osep, Patrick Dendorfer +4
Multi-Object Tracking (MOT) has been notoriously difficult to evaluate. Previous metrics overemphasize the importance of either detection or association. To address this, we presen…
Making a Case for 3D Convolutions for Object Segmentation in Videos
Sabarinath Mahadevan, Ali Athar, Aljoša Ošep +3
The task of object segmentation in videos is usually accomplished by processing appearance and motion information separately using standard 2D convolutional networks, followed by a…