841 citations · 7.7k across the 158 of their papers we have counts for
170 papers
TriO: Tri-Modal Unsupervised Occupancy World Model for Anything Perception
Quinlan Sykora, Sourav Biswas, Christopher Diehl +3
We present TriO, a multi-modal unsupervised world model that predicts 4D occupancy, obstacle segmentation, flow and LiDAR. In contrast to prior work, TriO utilizes three distinct s…
Diffusion-guided Generalizable Enhancer for Urban Scene Reconstruction
Henry Che, Jingkang Wang, Yun Chen +3
Urban scene reconstruction from real-world observations has emerged as a powerful tool for self-driving development and testing. While current neural rendering approaches achieve h…
Traffic Scenario Orchestration from Language via Constraint Satisfaction
Frieda Rong, Chris Zhang, Kelvin Wong +1
Autonomous vehicles (AVs) require extensive testing in simulation, but test case generation for driving scenarios is laborious. The desired scenarios are often out-of-distribution…
Conditional Flow-VAE for Safety-Critical Traffic Scenario Generation
Zimu Gong, Brian Zhaoning Zhang, Chris Zhang +2
Safety-critical scenarios are essential for the development of autonomous vehicles (AVs) but are rare in real-world driving data. While simulation offers a way to generate such sce…
GenAssets: Generating in-the-wild 3D Assets in Latent Space
Ze Yang, Jingkang Wang, Haowei Zhang +3
High-quality 3D assets for traffic participants are critical for multi-sensor simulation, which is essential for the safe end-to-end development of autonomy. Building assets from i…
Efficient Equivariant Transformer for Self-Driving Agent Modeling
Scott Xu, Dian Chen, Kelvin Wong +3
Accurately modeling agent behaviors is an important task in self-driving. It is also a task with many symmetries, such as equivariance to the order of agents and objects in the sce…
FOMO-3D: Using Vision Foundation Models for Long-Tailed 3D Object Detection
Anqi Joyce Yang, James Tu, Nikita Dvornik +2
In order to navigate complex traffic environments, self-driving vehicles must recognize many semantic classes pertaining to vulnerable road users or traffic control devices. Howeve…
Flux4D: Flow-based Unsupervised 4D Reconstruction
Jingkang Wang, Henry Che, Yun Chen +4
Reconstructing large-scale dynamic scenes from visual observations is a fundamental challenge in computer vision, with critical implications for robotics and autonomous systems. Wh…
SaLF: Sparse Local Fields for Multi-Sensor Rendering in Real-Time
Yun Chen, Matthew Haines, Jingkang Wang +5
High-fidelity sensor simulation of light-based sensors such as cameras and LiDARs is critical for safe and accurate autonomy testing. Neural radiance field (NeRF)-based methods tha…
G3R: Gradient Guided Generalizable Reconstruction
Yun Chen, Jingkang Wang, Ze Yang +2
Large scale 3D scene reconstruction is important for applications such as virtual reality and simulation. Existing neural rendering approaches (e.g., NeRF, 3DGS) have achieved real…
UniCal: Unified Neural Sensor Calibration
Ze Yang, George Chen, Haowei Zhang +5
Self-driving vehicles (SDVs) require accurate calibration of LiDARs and cameras to fuse sensor data accurately for autonomy. Traditional calibration methods typically leverage fidu…
Learning to Drive via Asymmetric Self-Play
Chris Zhang, Sourav Biswas, Kelvin Wong +5
Large-scale data is crucial for learning realistic and capable driving policies. However, it can be impractical to rely on scaling datasets with real data alone. The majority of dr…
DeTra: A Unified Model for Object Detection and Trajectory Forecasting
Sergio Casas, Ben Agro, Jiageng Mao +4
The tasks of object detection and trajectory forecasting play a crucial role in understanding the scene for autonomous driving. These tasks are typically executed in a cascading ma…
UnO: Unsupervised Occupancy Fields for Perception and Forecasting
Ben Agro, Quinlan Sykora, Sergio Casas +2
Perceiving the world and forecasting its future state is a critical task for self-driving. Supervised approaches leverage annotated object labels to learn a model of the world -- t…
QuAD: Query-based Interpretable Neural Motion Planning for Autonomous Driving
Sourav Biswas, Sergio Casas, Quinlan Sykora +3
A self-driving vehicle must understand its environment to determine the appropriate action. Traditional autonomy systems rely on object detection to find the agents in the scene. H…
LightSim: Neural Lighting Simulation for Urban Scenes
Ava Pun, Gary Sun, Jingkang Wang +5
Different outdoor illumination conditions drastically alter the appearance of urban scenes, and they can harm the performance of image-based robot perception systems if not seen du…
4D-Former: Multimodal 4D Panoptic Segmentation
Ali Athar, Enxu Li, Sergio Casas +1
4D panoptic segmentation is a challenging but practically useful task that requires every point in a LiDAR point-cloud sequence to be assigned a semantic class label, and individua…
Real-Time Neural Rasterization for Large Scenes
Jeffrey Yunfan Liu, Yun Chen, Ze Yang +3
We propose a new method for realistic real-time novel-view synthesis (NVS) of large scenes. Existing neural rendering methods generate realistic results, but primarily work for sma…
Reconstructing Objects in-the-wild for Realistic Sensor Simulation
Ze Yang, Sivabalan Manivasagam, Yun Chen +3
Reconstructing objects from real world data and rendering them at novel views is critical to bringing realism, diversity and scale to simulation for robotics training and testing.…
Towards Unsupervised Object Detection From LiDAR Point Clouds
Lunjun Zhang, Anqi Joyce Yang, Yuwen Xiong +4
In this paper, we study the problem of unsupervised object detection from 3D point clouds in self-driving scenes. We present a simple yet effective method that exploits (i) point c…
MemorySeg: Online LiDAR Semantic Segmentation with a Latent Memory
Enxu Li, Sergio Casas, Raquel Urtasun
Semantic segmentation of LiDAR point clouds has been widely studied in recent years, with most existing methods focusing on tackling this task using a single scan of the environmen…
UltraLiDAR: Learning Compact Representations for LiDAR Completion and Generation
Yuwen Xiong, Wei-Chiu Ma, Jingkang Wang +1
LiDAR provides accurate geometric measurements of the 3D world. Unfortunately, dense LiDARs are very expensive and the point clouds captured by low-beam LiDAR are often sparse. To…
CADSim: Robust and Scalable in-the-wild 3D Reconstruction for Controllable Sensor Simulation
Jingkang Wang, Sivabalan Manivasagam, Yun Chen +5
Realistic simulation is key to enabling safe and scalable development of % self-driving vehicles. A core component is simulating the sensors so that the entire autonomy system can…
Adv3D: Generating Safety-Critical 3D Objects through Closed-Loop Simulation
Jay Sarva, Jingkang Wang, James Tu +3
Self-driving vehicles (SDVs) must be rigorously tested on a wide range of scenarios to ensure safe deployment. The industry typically relies on closed-loop simulation to evaluate h…