12 papers · 1 filter
LIFT: Layout-In-Future Video Generation under Large Viewpoint Change via On-Policy Self-Distillation
Shengxiang Ji, Boyang Wang, Haiyang Xu +9
We introduce LIFT, a unified image-to-video generation framework that complements camera control with Layout-In-FuTure control, enabling users to specify what should appear in a fu…
VLGA: Vision-Language-Geometry-Action Models for Autonomous Driving
Jin Yao, Dhruva Dixith Kurra, Tom Lampo +3
Vision-language-action (VLA) models can describe scenes and reason about them in language, yet still struggle to ground their actions in the dense 3D world around them. Existing ap…
OmniShotCut: Holistic Relational Shot Boundary Detection with Shot-Query Transformer
Boyang Wang, Guangyi Xu, Jiahui Zhang +2
Shot Boundary Detection (SBD) aims to automatically identify shot changes and divide a video into coherent shots. While SBD was widely studied in the literature, existing methods o…
WildRayZer: Self-supervised Large View Synthesis in Dynamic Environments
Xuweiyi Chen, Wentao Zhou, Zezhou Cheng
We present WildRayZer, a self-supervised framework for novel view synthesis (NVS) in dynamic environments where both the camera and objects move. Dynamic content breaks the multi-v…
LabelAny3D: Label Any Object 3D in the Wild
Jin Yao, Radowan Mahmud Redoy, Sebastian Elbaum +2
Detecting objects in 3D space from monocular input is crucial for applications ranging from robotics to scene understanding. Despite advanced performance in the indoor and autonomo…
Stereo4DWalker: Learning 4D-aware Embodied Urban Navigation from Internet Stereo Videos
Wentao Zhou, Xuweiyi Chen, Vignesh Rajagopal +3
Despite rapid progress, embodied navigation in dynamic and unstructured urban environments remains brittle. Most existing approaches directly map monocular visual inputs to actions…
SAB3R: Semantic-Augmented Backbone in 3D Reconstruction
Xuweiyi Chen, Tian Xia, Sihan Xu +3
We introduce a new task, Map and Locate, which unifies the traditionally distinct objectives of open-vocabulary segmentation - detecting and segmenting object instances based on na…
Point-MoE: Large-Scale Multi-Dataset Training with Mixture-of-Experts for 3D Semantic Segmentation
Xuweiyi Chen, Wentao Zhou, Aruni RoyChowdhury +1
While massively scaling both data and models have become central in NLP and 2D vision, their benefits for 3D point cloud understanding remain limited. We study the initial step of…
Frame In-N-Out: Unbounded Controllable Image-to-Video Generation
Boyang Wang, Xuweiyi Chen, Matheus Gadelha +1
Controllability, temporal coherence, and detail synthesis remain the most critical challenges in video generation. In this paper, we focus on a commonly used yet underexplored cine…
Probing the Mid-level Vision Capabilities of Self-Supervised Learning
Xuweiyi Chen, Markus Marks, Zezhou Cheng
Mid-level vision capabilities - such as generic object localization and 3D geometric understanding - are not only fundamental to human vision but are also crucial for many real-wor…
Semantic-Free Procedural 3D Shapes Are Surprisingly Good Teachers
Xuweiyi Chen, Zezhou Cheng
Self-supervised learning has emerged as a promising approach for acquiring transferable 3D representations from unlabeled 3D point clouds. Unlike 2D images, which are widely access…
Open Vocabulary Monocular 3D Object Detection
Jin Yao, Hao Gu, Xuweiyi Chen +2
We propose and study open-vocabulary monocular 3D detection, a novel task that aims to detect objects of any categores in metric 3D space from a single RGB image. Existing 3D objec…