130 citations · 279 across the 41 of their papers we have counts for
49 papers
Foundation4D: Towards a unified perception and reconstruction model for autonomous vehicles
NVIDIA, :, Sérgio Agostinho +25
We introduce Foundation4D, a unified feed-forward model for geometry estimation, 3D detection, semantic occupancy, and Gaussian reconstruction from multi-camera driving video. A sh…
PixelUMM: Encoder-Free Unified Image and Video Understanding and Generation
Cong Wei, Xuanchi Ren, Bryan Chu +8
Unified Multimodal Models (UMMs) often rely on separate visual representations for understanding and generation, increasing visual context length and complicating integration with…
Unified Video Dense Prediction from Disjoint Data
Yihong Sun, Seoung Wug Oh, Jiahui Huang +2
Scene understanding requires simultaneous prediction about geometry, appearance, and semantics. However, existing task-specific annotations are fragmented across incompatible, doma…
Instant NuRec: Feed-Forward 3D Gaussian Reconstruction for Driving Scene Simulation
NVIDIA, :, Jiahui Huang +14
3D simulation platforms are critical for autonomous driving because they enable end-to-end policy evaluation, thereby reducing development costs and improving safety. In recent yea…
MT-EditFlow: Reinforcement Learning for Multi-Turn Image Editing with Flow Matching
Jiahui Huang, Yasi Zhang, Tianyu Chen +6
Recent breakthroughs in instruction-based image editing have captured significant attention, as models are now capable of handling real-world editing demands with the practicality…
Déjà View: Looping Transformers for Multi-View 3D Reconstruction
Alessandro Burzio, Tobias Fischer, Sven Elflein +9
Recent feed-forward 3D reconstruction transformers have scaled to over a billion parameters, following the broader trend of increasing model capacity in computer vision. Yet emergi…
Xiaomi OneVL: One-Step Latent Reasoning and Planning with Vision-Language Explanation
Jinghui Lu, Jiayi Guan, Zhijian Huang +47
Chain-of-Thought (CoT) reasoning has become a powerful driver of trajectory prediction in VLA-based autonomous driving, yet its autoregressive nature imposes a latency cost that is…
Asset Harvester: Extracting 3D Assets from Autonomous Driving Logs for Simulation
Tianshi Cao, Jiawei Ren, Yuxuan Zhang +12
Closed-loop simulation is a core component of autonomous vehicle (AV) development, enabling scalable testing, training, and safety validation before real-world deployment. Neural s…
TokenGS: Decoupling 3D Gaussian Prediction from Pixels with Learnable Tokens
Jiawei Ren, Michal Jan Tyszkiewicz, Jiahui Huang +1
In this work, we revisit several key design choices of modern Transformer-based approaches for feed-forward 3D Gaussian Splatting (3DGS) prediction. We argue that the common practi…
Lyra 2.0: Explorable Generative 3D Worlds
Tianchang Shen, Sherwin Bahmani, Kai He +12
Recent advances in video generation enable a new paradigm for 3D scene creation: generating camera-controlled videos that simulate scene walkthroughs, then lifting them to 3D via f…
DAGE: Dual-Stream Architecture for Efficient and Fine-Grained Geometry Estimation
Tuan Duc Ngo, Jiahui Huang, Seoung Wug Oh +4
Estimating accurate, view-consistent geometry and camera poses from uncalibrated multi-view/video inputs remains challenging - especially at high spatial resolutions and over long…
Depth Completion as Parameter-Efficient Test-Time Adaptation
Bingxin Ke, Qunjie Zhou, Jiahui Huang +5
We introduce CAPA, a parameter-efficient test-time optimization framework that adapts pre-trained 3D foundation models (FMs) for depth completion, using sparse geometric cues. Unli…
VideoMaMa: Mask-Guided Video Matting via Generative Prior
Sangbeom Lim, Seoung Wug Oh, Jiahui Huang +3
Generalizing video matting models to real-world videos remains a significant challenge due to the scarcity of labeled data. To address this, we present Video Mask-to-Matte Model (V…
The Dynamic Prior: Understanding 3D Structures for Casual Dynamic Videos
Zhuoyuan Wu, Xurui Yang, Jiahui Huang +2
Estimating accurate camera poses, 3D scene geometry, and object motion from in-the-wild videos is a long-standing challenge for classical structure from motion pipelines due to the…
NeuralSSD: A Neural Solver for Signed Distance Surface Reconstruction
Zi-Chen Xi, Jiahui Huang, Hao-Xiang Chen +4
We proposed a generalized method, NeuralSSD, for reconstructing a 3D implicit surface from the widely-available point cloud data. NeuralSSD is a solver-based on the neural Galerkin…
Lyra: Generative 3D Scene Reconstruction via Video Diffusion Model Self-Distillation
Sherwin Bahmani, Tianchang Shen, Jiawei Ren +10
The ability to generate virtual environments is crucial for applications ranging from gaming to physical AI domains such as robotics, autonomous driving, and industrial AI. Current…
ViPE: Video Pose Engine for 3D Geometric Perception
Jiahui Huang, Qunjie Zhou, Hesam Rabeti +12
Accurate 3D geometric perception is an important prerequisite for a wide range of spatial AI systems. While state-of-the-art methods depend on large-scale training data, acquiring…
AnthroTAP: Learning Point Tracking with Real-World Motion
Inès Hyeonsu Kim, Seokju Cho, Jahyeok Koo +5
Point tracking models often struggle to generalize to real-world videos because large-scale training data is predominantly synthetic$\unicode{x2014}$the only source currently feasi…
FRAME: Pre-Training Video Feature Representations via Anticipation and Memory
Sethuraman TV, Savya Khosla, Vignesh Srinivasakumar +5
Dense video prediction tasks, such as object tracking and semantic segmentation, require video encoders that generate temporally consistent, spatially dense features for every fram…
Seurat: From Moving Points to Depth
Seokju Cho, Jiahui Huang, Seungryong Kim +1
Accurate depth estimation from monocular videos remains challenging due to ambiguities inherent in single-view geometry, as crucial depth cues like stereopsis are absent. However,…
VideoPanda: Video Panoramic Diffusion with Multi-view Attention
Kevin Xie, Amirmojtaba Sabour, Jiahui Huang +5
High resolution panoramic video content is paramount for immersive experiences in Virtual Reality, but is non-trivial to collect as it requires specialized equipment and intricate…
GEN3C: 3D-Informed World-Consistent Video Generation with Precise Camera Control
Xuanchi Ren, Tianchang Shen, Jiahui Huang +7
We present GEN3C, a generative video model with precise Camera Control and temporal 3D Consistency. Prior video models already generate realistic videos, but they tend to leverage…
MotionCanvas: Cinematic Shot Design with Controllable Image-to-Video Generation
Jinbo Xing, Long Mai, Cusuh Ham +5
This paper presents a method that allows users to design cinematic video shots in the context of image-to-video generation. Shot design, a critical aspect of filmmaking, involves m…
Exploring Temporally-Aware Features for Point Tracking
Inès Hyeonsu Kim, Seokju Cho, Jiahui Huang +3
Point tracking in videos is a fundamental task with applications in robotics, video editing, and more. While many vision tasks benefit from pre-trained feature backbones to improve…