4 citations · 6 across the 8 of their papers we have counts for
10 papers
OmniTaskonomy: When Does Visual Generation Improve Visual Understanding?
Jiaxin Ge, Yiming Qin, Ji Xie +13
Training a model to generate visual content can encourage it to learn rich perceptual capabilities related to geometry, spatial relationships, and objectness; yet, its benefits for…
Tracking by Predicting 3-D Gaussians Over Time
Tanish Baranwal, Himanshu Gaurav Singh, Jathushan Rajasegaran +1
We propose Video Gaussian Masked Autoencoders (Video-GMAE), a self-supervised approach for representation learning that encodes a sequence of images into a set of Gaussian splats m…
FoundationMotion: Auto-Labeling and Reasoning about Spatial Movement in Videos
Yulu Gan, Ligeng Zhu, Dandan Shan +8
Motion understanding is fundamental to physical reasoning, enabling models to infer dynamics and predict future states. However, state-of-the-art models still struggle on recent mo…
Poly-Autoregressive Prediction for Modeling Interactions
Neerja Thakkar, Tara Sadjadpour, Jathushan Rajasegaran +2
We introduce a simple framework for predicting the behavior of an agent in multi-agent settings. In contrast to autoregressive (AR) tasks, such as language processing, our focus is…
An Empirical Study of Autoregressive Pre-training from Videos
Jathushan Rajasegaran, Ilija Radosavovic, Rahul Ravishankar +3
We empirically study autoregressive pre-training from videos. To perform our study, we construct a series of autoregressive video models, called Toto. We treat videos as sequences…
Gaussian Masked Autoencoders
Jathushan Rajasegaran, Xinlei Chen, Rulilong Li +3
This paper explores Masked Autoencoders (MAE) with Gaussian Splatting. While reconstructive self-supervised learning frameworks such as MAE learns good semantic abstractions, it is…
Scaling Properties of Diffusion Models for Perceptual Tasks
Rahul Ravishankar, Zeeshan Patel, Jathushan Rajasegaran +1
In this paper, we argue that iterative computation with diffusion models offers a powerful paradigm for not only generation but also visual perception tasks. We unify tasks such as…
Wolf: Dense Video Captioning with a World Summarization Framework
Boyi Li, Ligeng Zhu, Ran Tian +20
We propose Wolf, a WOrLd summarization Framework for accurate video captioning. Wolf is an automated captioning framework that adopts a mixture-of-experts approach, leveraging comp…
Synthesizing Moving People with 3D Control
Boyi Li, Junming Chen, Jathushan Rajasegaran +3
In this paper, we present a diffusion model-based framework for animating people from a single image for a given target 3D motion sequence. Our approach has two core components: a)…
Interactive Task Planning with Language Models
Boyi Li, Philipp Wu, Pieter Abbeel +1
An interactive robot framework accomplishes long-horizon task planning and can easily generalize to new goals and distinct tasks, even during execution. However, most traditional m…