16 papers
ActWorld: From Explorable to Interactive World Model via Action-Aware Memory
Zhexiao Xiong, Yizhi Song, Hao Kang +11
Interactive world models aim to simulate environment dynamics under real-time user actions. However, their action vocabulary is largely confined to navigation: most actions corresp…
Track2View: 4D-Consistent Camera-Controlled Video Generation via Paired 3D Point Tracks
Feng Qiao, Zhaochong An, Zhexiao Xiong +2
Re-rendering an existing video from a novel camera viewpoint requires the output to follow the prescribed camera trajectory while preserving the appearance and dynamics of the orig…
GenOpticalFlow: A Generative Approach to Unsupervised Optical Flow Learning
Yixuan Luo, Feng Qiao, Zhexiao Xiong +2
Optical flow estimation is a fundamental problem in computer vision, yet the reliance on expensive ground-truth annotations limits the scalability of supervised approaches. Althoug…
Reconstruction Matters: Learning Geometry-Aligned BEV Representation through 3D Gaussian Splatting
Yiren Lu, Xin Ye, Burhaneddin Yaman +4
Bird's-Eye-View (BEV) perception serves as a cornerstone for autonomous driving, offering a unified spatial representation that fuses surrounding-view images to enable reasoning fo…
PhysAlign: Physics-Coherent Image-to-Video Generation through Feature and 3D Representation Alignment
Zhexiao Xiong, Yizhi Song, Liu He +4
Video Diffusion Models (VDMs) offer a promising approach for simulating dynamic scenes and environments, with broad applications in robotics and media generation. However, existing…
UniDrive-WM: Unified Understanding, Planning and Generation World Model for Autonomous Driving
Zhexiao Xiong, Xin Ye, Burhan Yaman +5
World models have become central to autonomous driving, where accurate scene understanding and future prediction are crucial for safe control. Recent work has explored using vision…
PanoDreamer: Consistent Text to 360-Degree Scene Generation
Zhexiao Xiong, Zhang Chen, Zhong Li +2
Automatically generating a complete 3D scene from a text description, a reference image, or both has significant applications in fields like virtual reality and gaming. However, cu…
DeclutterNeRF: Generative-Free 3D Scene Recovery for Occlusion Removal
Wanzhou Liu, Zhexiao Xiong, Xinyu Li +1
Recent novel view synthesis (NVS) techniques, including Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS) have greatly advanced 3D scene reconstruction with high-quali…
Towards Open-World Generation of Stereo Images and Unsupervised Matching
Feng Qiao, Zhexiao Xiong, Eric Xing +1
Stereo images are fundamental to numerous applications, including extended reality (XR) devices, autonomous driving, and robotics. Unfortunately, acquiring high-quality stereo imag…
GroundingBooth: Grounding Text-to-Image Customization
Zhexiao Xiong, Wei Xiong, Jing Shi +3
Recent approaches in text-to-image customization have primarily focused on preserving the identity of the input subject, but often fail to control the spatial location and size of…
PSM: Learning Probabilistic Embeddings for Multi-scale Zero-Shot Soundscape Mapping
Subash Khanal, Eric Xing, Srikumar Sastry +4
A soundscape is defined by the acoustic environment a person perceives at a location. In this work, we propose a framework for mapping soundscapes across the Earth. Since soundscap…
MCPDepth: Omnidirectional Depth Estimation via Stereo Matching from Multi-Cylindrical Panoramas
Feng Qiao, Zhexiao Xiong, Xinge Zhu +3
Omnidirectional depth estimation presents a significant challenge due to the inherent distortions in panoramic images. Despite notable advancements, the impact of projection method…
Mixed-View Panorama Synthesis using Geospatially Guided Diffusion
Zhexiao Xiong, Xin Xing, Scott Workman +2
We introduce the task of mixed-view panorama synthesis, where the goal is to synthesize a novel panorama given a small set of input panoramas and a satellite image of the area. Thi…
Eroding Trust In Aerial Imagery: Comprehensive Analysis and Evaluation Of Adversarial Attacks In Geospatial Systems
Michael Lanier, Aayush Dhakal, Zhexiao Xiong +3
In critical operations where aerial imagery plays an essential role, the integrity and trustworthiness of data are paramount. The emergence of adversarial attacks, particularly tho…
Vision-Language Pseudo-Labels for Single-Positive Multi-Label Learning
Xin Xing, Zhexiao Xiong, Abby Stylianou +3
This paper presents a novel approach to Single-Positive Multi-label Learning. In general multi-label learning, a model learns to predict multiple labels or categories for a single…
StereoFlowGAN: Co-training for Stereo and Flow with Unsupervised Domain Adaptation
Zhexiao Xiong, Feng Qiao, Yu Zhang +1
We introduce a novel training strategy for stereo matching and optical flow estimation that utilizes image-to-image translation between synthetic and real image domains. Our approa…