95 citations · 502 across the 34 of their papers we have counts for
40 papers
Parallel Sequence Modeling via Generalized Spatial Propagation Network
Hongjun Wang, Wonmin Byeon, Jiarui Xu +6
We present the Generalized Spatial Propagation Network (GSPN), a new attention mechanism optimized for vision tasks that inherently captures 2D spatial structures. Existing attenti…
NaVILA: Legged Robot Vision-Language-Action Model for Navigation
An-Chieh Cheng, Yandong Ji, Zhaojing Yang +7
This paper proposes to solve the problem of Vision-and-Language Navigation with legged robots, which not only provides a flexible way for humans to command but also allows the robo…
HOVER: Versatile Neural Whole-Body Controller for Humanoid Robots
Tairan He, Wenli Xiao, Toru Lin +9
Humanoid whole-body control requires adapting to diverse tasks such as navigation, loco-manipulation, and tabletop manipulation, each demanding a different mode of control. For exa…
SpatialRGPT: Grounded Spatial Reasoning in Vision Language Models
An-Chieh Cheng, Hongxu Yin, Yang Fu +5
Vision Language Models (VLMs) have demonstrated remarkable performance in 2D vision and language tasks. However, their ability to reason about spatial arrangements remains limited.…
COLMAP-Free 3D Gaussian Splatting
Yang Fu, Sifei Liu, Amey Kulkarni +3
While neural rendering has led to impressive advances in scene reconstruction and novel view synthesis, it relies heavily on accurately pre-computed camera poses. To relax this con…
3D Reconstruction with Generalizable Neural Fields using Scene Priors
Yang Fu, Shalini De Mello, Xueting Li +4
High-fidelity 3D scene reconstruction has been substantially advanced by recent progress in neural fields. However, most existing methods train a separate network from scratch for…
Zero-shot Pose Transfer for Unrigged Stylized 3D Characters
Jiashun Wang, Xueting Li, Sifei Liu +4
Transferring the pose of a reference avatar to stylized 3D characters of various shapes is a fundamental task in computer graphics. Existing methods either require the stylized cha…
CoordGAN: Self-Supervised Dense Correspondences Emerge from GANs
Jiteng Mu, Shalini De Mello, Zhiding Yu +4
Recent advances show that Generative Adversarial Networks (GANs) can synthesize images with smooth variations along semantically meaningful latent directions, such as pose, express…
GroupViT: Semantic Segmentation Emerges from Text Supervision
Jiarui Xu, Shalini De Mello, Sifei Liu +4
Grouping and recognition are important components of visual scene understanding, e.g., for object detection and semantic segmentation. With end-to-end deep learning systems, groupi…
Learning Continuous Environment Fields via Implicit Functions
Xueting Li, Shalini De Mello, Xiaolong Wang +3
We propose a novel scene representation that encodes reaching distance -- the distance between any position in the scene to a goal along a feasible trajectory. We demonstrate that…
Self-Supervised Object Detection via Generative Image Synthesis
Siva Karthik Mustikovela, Shalini De Mello, Aayush Prakash +5
We present SSOD, the first end-to-end analysis-by synthesis framework with controllable GANs for the task of self-supervised object detection. We use collections of real world imag…
Learning Contrastive Representation for Semantic Correspondence
Taihong Xiao, Sifei Liu, Shalini De Mello +3
Dense correspondence across semantically related images has been extensively studied, but still faces two challenges: 1) large variations in appearance, scale and pose exist even f…
Weakly-Supervised Physically Unconstrained Gaze Estimation
Rakshit Kothari, Shalini De Mello, Umar Iqbal +3
A major challenge for physically unconstrained gaze estimation is acquiring training data with 3D gaze annotations for in-the-wild and outdoor scenarios. In contrast, videos of hum…
Learning to Track Instances without Video Annotations
Yang Fu, Sifei Liu, Umar Iqbal +3
Tracking segmentation masks of multiple instances has been intensively studied, but still faces two fundamental challenges: 1) the requirement of large-scale, frame-wise annotation…
Online Adaptation for Consistent Mesh Reconstruction in the Wild
Xueting Li, Sifei Liu, Shalini De Mello +4
This paper presents an algorithm to reconstruct temporally consistent 3D meshes of deformable object instances from videos in the wild. Without requiring annotations of 3D mesh, 2D…
Improving Deep Stereo Network Generalization with Geometric Priors
Jialiang Wang, Varun Jampani, Deqing Sun +3
End-to-end deep learning methods have advanced stereo vision in recent years and obtained excellent results when the training and test data are similar. However, large datasets of…
Self-Supervised Viewpoint Learning From Image Collections
Siva Karthik Mustikovela, Varun Jampani, Shalini De Mello +4
Training deep neural networks to estimate the viewpoint of objects requires large labeled training datasets. However, manually labeling viewpoints is notoriously hard, error-prone,…
Self-supervised Single-view 3D Reconstruction via Semantic Consistency
Xueting Li, Sifei Liu, Kihwan Kim +4
We learn a self-supervised, single-view 3D reconstruction model that predicts the 3D mesh shape, texture and camera pose of a target object with a collection of 2D images and silho…
Exploiting Semantics for Face Image Deblurring
Ziyi Shen, Wei-Sheng Lai, Tingfa Xu +2
In this paper, we propose an effective and efficient face deblurring algorithm by exploiting semantic cues via deep convolutional neural networks. As the human faces are highly str…
SENSE: a Shared Encoder Network for Scene-flow Estimation
Huaizu Jiang, Deqing Sun, Varun Jampani +3
We introduce a compact network for holistic scene flow estimation, called SENSE, which shares common encoder features among four closely-related tasks: optical flow estimation, dis…
Joint-task Self-supervised Learning for Temporal Correspondence
Xueting Li, Sifei Liu, Shalini De Mello +3
This paper proposes to learn reliable dense correspondence from videos in a self-supervised manner. Our learning process integrates two highly related tasks: tracking large image r…
Learning Propagation for Arbitrarily-structured Data
Sifei Liu, Xueting Li, Varun Jampani +2
Processing an input signal that contains arbitrary structures, e.g., superpixels and point clouds, remains a big challenge in computer vision. Linear diffusion, an effective model…
Video Stitching for Linear Camera Arrays
Wei-Sheng Lai, Orazio Gallo, Jinwei Gu +3
Despite the long history of image and video stitching research, existing academic and commercial solutions still produce strong artifacts. In this work, we propose a wide-baseline…
Unsupervised Video Interpolation Using Cycle Consistency
Fitsum A. Reda, Deqing Sun, Aysegul Dundar +6
Learning to synthesize high frame rate videos via interpolation requires large quantities of high frame rate training videos, which, however, are scarce, especially at high resolut…