84 citations · 150 across the 26 of their papers we have counts for
31 papers
A Phenomenology of Neural Geometry in Vision-Language-Action Models
Mozes Jacobs, Aiden Swann, Mathilde Papillon +16
Robotic manipulation is inherently geometric, but what shape does the world take inside the policy that performs it? Prior work on vision-language-action (VLA) policies establishes…
Language Model Activations Inhabit Privileged Error-Correcting Basins
Matthew Finlayson, Francisco Pernice, Eric Todd +16
Language models exhibit remarkable robustness, continuing to produce coherent text even when their activations are perturbed by interventions like linear steering. We hypothesize t…
Token-Shuffle: Towards High-Resolution Image Generation with Autoregressive Models
Xu Ma, Peize Sun, Haoyu Ma +22
Autoregressive (AR) models, long dominant in language generation, are increasingly applied to image synthesis but are often considered less competitive than Diffusion-based models.…
HOIGPT: Learning Long Sequence Hand-Object Interaction with Language Models
Mingzhen Huang, Fu-Jen Chu, Bugra Tekin +10
We introduce HOIGPT, a token-based generative method that unifies 3D hand-object interactions (HOI) perception and generation, offering the first comprehensive solution for caption…
Fast3R: Towards 3D Reconstruction of 1000+ Images in One Forward Pass
Jianing Yang, Alexander Sax, Kevin J. Liang +6
Multi-view 3D reconstruction remains a core challenge in computer vision, particularly in applications requiring accurate and scalable representations across diverse perspectives.…
VideoLifter: Lifting Videos to 3D with Fast Hierarchical Stereo Alignment
Wenyan Cong, Hanqing Zhu, Kevin Wang +6
Efficiently reconstructing 3D scenes from monocular video remains a core challenge in computer vision, vital for applications in virtual reality, robotics, and scene understanding.…
OmniPose6D: Towards Short-Term Object Pose Tracking in Dynamic Scenes from Monocular RGB
Yunzhi Lin, Yipu Zhao, Fu-Jen Chu +6
To address the challenge of short-term object pose tracking in dynamic environments with monocular RGB input, we introduce a large-scale synthetic dataset OmniPose6D, crafted to mi…
ADen: Adaptive Density Representations for Sparse-view Camera Pose Estimation
Hao Tang, Weiyao Wang, Pierre Gleize +1
Recovering camera poses from a set of images is a foundational task in 3D computer vision, which powers key applications such as 3D scene/object reconstructions. Classic methods of…
3x2: 3D Object Part Segmentation by 2D Semantic Correspondences
Anh Thai, Weiyao Wang, Hao Tang +3
3D object part segmentation is essential in computer vision applications. While substantial progress has been made in 2D object part segmentation, the 3D counterpart has received l…
ICON: Incremental CONfidence for Joint Pose and Radiance Field Optimization
Weiyao Wang, Pierre Gleize, Hao Tang +3
Neural Radiance Fields (NeRF) exhibit remarkable performance for Novel View Synthesis (NVS) given a set of 2D images. However, NeRF training requires accurate camera pose for each…
NOVIS: A Case for End-to-End Near-Online Video Instance Segmentation
Tim Meinhardt, Matt Feiszli, Yuchen Fan +2
Until recently, the Video Instance Segmentation (VIS) community operated under the common belief that offline methods are generally superior to a frame by frame online processing.…
SiLK -- Simple Learned Keypoints
Pierre Gleize, Weiyao Wang, Matt Feiszli
Keypoint detection & descriptors are foundational tech-nologies for computer vision tasks like image matching, 3D reconstruction and visual odometry. Hand-engineered methods like H…
MINOTAUR: Multi-task Video Grounding From Multimodal Queries
Raghav Goyal, Effrosyni Mavroudi, Xitong Yang +5
Video understanding tasks take many forms, from action detection to visual query localization and spatio-temporal grounding of sentences. These tasks differ in the type of inputs (…
EgoTracks: A Long-term Egocentric Visual Object Tracking Dataset
Hao Tang, Kevin Liang, Matt Feiszli +1
Visual object tracking is a key component to many egocentric vision problems. However, the full spectrum of challenges of egocentric tracking faced by an embodied AI is underrepres…
Open-World Instance Segmentation: Exploiting Pseudo Ground Truth From Learned Pairwise Affinity
Weiyao Wang, Matt Feiszli, Heng Wang +2
Open-world instance segmentation is the task of grouping pixels into object instances without any pre-determined taxonomy. This is challenging, as state-of-the-art methods rely on…
GEB+: A Benchmark for Generic Event Boundary Captioning, Grounding and Retrieval
Yuxuan Wang, Difei Gao, Licheng Yu +3
Cognitive science has shown that humans perceive videos in terms of events separated by the state changes of dominant subjects. State changes trigger new events and are one of the…
PyTorchVideo: A Deep Learning Library for Video Understanding
Haoqi Fan, Tullie Murrell, Heng Wang +13
We introduce PyTorchVideo, an open-source deep-learning library that provides a rich set of modular, efficient, and reproducible components for a variety of video understanding tas…
Searching for Two-Stream Models in Multivariate Space for Video Recognition
Xinyu Gong, Heng Wang, Zheng Shou +3
Conventional video models rely on a single stream to capture the complex spatial-temporal features. Recent work on two-stream video models, such as SlowFast network and AssembleNet…
Unidentified Video Objects: A Benchmark for Dense, Open-World Segmentation
Weiyao Wang, Matt Feiszli, Heng Wang +1
Current state-of-the-art object detection and segmentation methods work well under the closed-world assumption. This closed-world setting assumes that the list of object categories…
Generic Event Boundary Detection: A Benchmark for Event Segmentation
Mike Zheng Shou, Stan Weixian Lei, Weiyao Wang +2
This paper presents a novel task together with a new benchmark for detecting generic, taxonomy-free event boundaries that segment a whole video into chunks. Conventional work in te…
FP-NAS: Fast Probabilistic Neural Architecture Search
Zhicheng Yan, Xiaoliang Dai, Peizhao Zhang +3
Differential Neural Architecture Search (NAS) requires all layer choices to be held in memory simultaneously; this limits the size of both search space and final architecture. In c…
SF-Net: Single-Frame Supervision for Temporal Action Localization
Fan Ma, Linchao Zhu, Yi Yang +4
In this paper, we study an intermediate form of supervision, i.e., single-frame supervision, for temporal action localization (TAL). To obtain the single-frame supervision, the ann…
Don't Judge an Object by Its Context: Learning to Overcome Contextual Bias
Krishna Kumar Singh, Dhruv Mahajan, Kristen Grauman +3
Existing models often leverage co-occurrences between objects and their context to improve recognition accuracy. However, strongly relying on context risks a model's generalizabili…
Only Time Can Tell: Discovering Temporal Data for Temporal Modeling
Laura Sevilla-Lara, Shengxin Zha, Zhicheng Yan +3
Understanding temporal information and how the visual world changes over time is a fundamental ability of intelligent systems. In video understanding, temporal information is at th…