641 citations · 2.3k across the 122 of their papers we have counts for
132 papers
CLEAR: Closed-Loop Reinforcement Learning at Scale for End-to-End Autonomous Driving
Yunxiao Shi, Hong Cai, Mohammad Ghavamzadeh +1
End-to-end autonomous driving (E2E-AD) aims to directly map raw sensor information to driving actions. Recently, with the rapid advancement of multi-modal large language models (ML…
SciFlow: Semantic Cross Interference for Self-Supervised Optical Flow Domain Generalization
Jamie Menjay Lin, Jisoo Jeong, Hong Cai +2
Motions of objects and scenes carry essential intelligence in video understanding, offering rich cues for interpreting dynamic settings and interactions. Due to the cost and scarci…
Neodragon: Mobile Video Generation using Diffusion Transformer
Animesh Karnewar, Denis Korzhenkov, Ioannis Lelekas +10
We introduce Neodragon, a text-to-video system capable of generating 2s (49 frames @24 fps) videos at the 640x1024 resolution directly on a Qualcomm Hexagon NPU in a record 6.7s (7…
ODG: Occupancy Prediction Using Dual Gaussians
Yunxiao Shi, Yinhao Zhu, Shizhong Han +4
Occupancy prediction infers fine-grained 3D geometry and semantics from camera images of the surrounding environment, making it a critical perception task for autonomous driving. E…
DySS: Dynamic Queries and State-Space Learning for Efficient 3D Object Detection from Multi-Camera Videos
Rajeev Yasarla, Shizhong Han, Hong Cai +1
Camera-based 3D object detection in Bird's Eye View (BEV) is one of the most important perception tasks in autonomous driving. Earlier methods rely on dense BEV features, which are…
Learning Optical Flow Field via Neural Ordinary Differential Equation
Leyla Mirvakhabova, Hong Cai, Jisoo Jeong +3
Recent works on optical flow estimation use neural networks to predict the flow field that maps positions of one image to positions of the other. These networks consist of a featur…
RoCA: Robust Cross-Domain End-to-End Autonomous Driving
Rajeev Yasarla, Shizhong Han, Hsin-Pai Cheng +7
End-to-end (E2E) autonomous driving has recently emerged as a new paradigm, offering significant potential. However, few studies have looked into the practical challenge of deploym…
BePo: Dual Representation for 3D Occupancy Prediction
Yunxiao Shi, Hong Cai, Jisoo Jeong +4
3D occupancy infers fine-grained 3D geometry and semantics which is critical for autonomous driving. Most existing approaches carry high compute costs, requiring dense 3D feature v…
FALO: Fast and Accurate LiDAR 3D Object Detection on Resource-Constrained Devices
Shizhong Han, Hsin-Pai Cheng, Hong Cai +3
Existing LiDAR 3D object detection methods predominantely rely on sparse convolutions and/or transformers, which can be challenging to run on resource-constrained edge devices, due…
Improving Optical Flow and Stereo Depth Estimation by Leveraging Uncertainty-Based Learning Difficulties
Jisoo Jeong, Hong Cai, Jamie Menjay Lin +1
Conventional training for optical flow and stereo depth models typically employs a uniform loss function across all pixels. However, this one-size-fits-all approach often overlooks…
H3O: Hyper-Efficient 3D Occupancy Prediction with Heterogeneous Supervision
Yunxiao Shi, Hong Cai, Amin Ansari +1
3D occupancy prediction has recently emerged as a new paradigm for holistic 3D scene understanding and provides valuable information for downstream planning in autonomous driving.…
SubZero: Composing Subject, Style, and Action via Zero-Shot Personalization
Shubhankar Borse, Kartikeya Bhardwaj, Mohammad Reza Karimi Dastjerdi +8
Diffusion models are increasingly popular for generative tasks, including personalized composition of subjects and styles. While diffusion models can generate user-specified subjec…
Distilling Multi-modal Large Language Models for Autonomous Driving
Deepti Hegde, Rajeev Yasarla, Hong Cai +7
Autonomous driving demands safe motion planning, especially in critical "long-tail" scenarios. Recent end-to-end autonomous driving systems leverage large language models (LLMs) as…
MoViE: Mobile Diffusion for Video Editing
Adil Karjauv, Noor Fathima, Ioannis Lelekas +3
Recent progress in diffusion-based video editing has shown remarkable potential for practical applications. However, these methods remain prohibitively expensive and challenging to…
Planar Gaussian Splatting
Farhad G. Zanjani, Hong Cai, Hanno Ackermann +2
This paper presents Planar Gaussian Splatting (PGS), a novel neural rendering approach to learn the 3D geometry and parse the 3D planes of a scene, directly from multiple RGB image…
HyperNet Fields: Efficiently Training Hypernetworks without Ground Truth by Learning Weight Trajectories
Eric Hedlin, Munawar Hayat, Fatih Porikli +2
To efficiently adapt large models or to train generative models of neural representations, Hypernetworks have drawn interest. While hypernetworks work well, training them is cumber…
DDIL: Diversity Enhancing Diffusion Distillation With Imitation Learning
Risheek Garrepalli, Shweta Mahajan, Munawar Hayat +1
Diffusion models excel at generative modeling (e.g., text-to-image) but sampling requires multiple denoising network passes, limiting practicality. Efforts such as progressive dist…
PADRe: A Unifying Polynomial Attention Drop-in Replacement for Efficient Vision Transformer
Pierre-David Letourneau, Manish Kumar Singh, Hsin-Pai Cheng +6
We present Polynomial Attention Drop-in Replacement (PADRe), a novel and unifying framework designed to replace the conventional self-attention mechanism in transformer models. Not…
ToSA: Token Selective Attention for Efficient Vision Transformers
Manish Kumar Singh, Rajeev Yasarla, Hong Cai +2
In this paper, we propose a novel token selective attention approach, ToSA, which can identify tokens that need to be attended as well as those that can skip a transformer layer. M…
FouRA: Fourier Low Rank Adaptation
Shubhankar Borse, Shreya Kadambi, Nilesh Prasad Pandey +7
While Low-Rank Adaptation (LoRA) has proven beneficial for efficiently fine-tuning large models, LoRA fine-tuned text-to-image diffusion models lack diversity in the generated imag…
SciFlow: Empowering Lightweight Optical Flow Models with Self-Cleaning Iterations
Jamie Menjay Lin, Jisoo Jeong, Hong Cai +3
Optical flow estimation is crucial to a variety of vision tasks. Despite substantial recent advancements, achieving real-time on-device optical flow estimation remains a complex ch…
OCAI: Improving Optical Flow Estimation by Occlusion and Consistency Aware Interpolation
Jisoo Jeong, Hong Cai, Risheek Garrepalli +3
The scarcity of ground-truth labels poses one major challenge in developing optical flow estimation models that are both generalizable and robust. While current methods rely on dat…
DeCoTR: Enhancing Depth Completion with 2D and 3D Attentions
Yunxiao Shi, Manish Kumar Singh, Hong Cai +1
In this paper, we introduce a novel approach that harnesses both 2D and 3D attentions to enable highly accurate depth completion without requiring iterative spatial propagations. S…
PosSAM: Panoptic Open-vocabulary Segment Anything
Vibashan VS, Shubhankar Borse, Hyojin Park +4
In this paper, we introduce an open-vocabulary panoptic segmentation model that effectively unifies the strengths of the Segment Anything Model (SAM) with the vision-language CLIP…