105 citations · 295 across the 23 of their papers we have counts for
27 papers
Tracking by Predicting 3-D Gaussians Over Time
Tanish Baranwal, Himanshu Gaurav Singh, Jathushan Rajasegaran +1
We propose Video Gaussian Masked Autoencoders (Video-GMAE), a self-supervised approach for representation learning that encodes a sequence of images into a set of Gaussian splats m…
Perception Encoder: The best visual embeddings are not at the output of the network
Daniel Bolya, Po-Yao Huang, Peize Sun +15
We introduce Perception Encoder (PE), a state-of-the-art vision encoder for image and video understanding trained via simple vision-language learning. Traditionally, vision encoder…
Poly-Autoregressive Prediction for Modeling Interactions
Neerja Thakkar, Tara Sadjadpour, Jathushan Rajasegaran +2
We introduce a simple framework for predicting the behavior of an agent in multi-agent settings. In contrast to autoregressive (AR) tasks, such as language processing, our focus is…
An Empirical Study of Autoregressive Pre-training from Videos
Jathushan Rajasegaran, Ilija Radosavovic, Rahul Ravishankar +3
We empirically study autoregressive pre-training from videos. To perform our study, we construct a series of autoregressive video models, called Toto. We treat videos as sequences…
Gaussian Masked Autoencoders
Jathushan Rajasegaran, Xinlei Chen, Rulilong Li +3
This paper explores Masked Autoencoders (MAE) with Gaussian Splatting. While reconstructive self-supervised learning frameworks such as MAE learns good semantic abstractions, it is…
Scaling Properties of Diffusion Models for Perceptual Tasks
Rahul Ravishankar, Zeeshan Patel, Jathushan Rajasegaran +1
In this paper, we argue that iterative computation with diffusion models offers a powerful paradigm for not only generation but also visual perception tasks. We unify tasks such as…
Synergy and Synchrony in Couple Dances
Vongani Maluleke, Lea Müller, Jathushan Rajasegaran +4
This paper asks to what extent social interaction influences one's behavior. We study this in the setting of two dancers dancing as a couple. We first consider a baseline in which…
FewShotNeRF: Meta-Learning-based Novel View Synthesis for Rapid Scene-Specific Adaptation
Piraveen Sivakumar, Paul Janson, Jathushan Rajasegaran +1
In this paper, we address the challenge of generating novel views of real-world objects with limited multi-view images through our proposed approach, FewShotNeRF. Our method utiliz…
EgoPet: Egomotion and Interaction Data from an Animal's Perspective
Amir Bar, Arya Bakhtiar, Danny Tran +5
Animals perceive the world to plan their actions and interact with other agents to accomplish complex tasks, demonstrating capabilities that are still unmatched by AI systems. To a…
Humanoid Locomotion as Next Token Prediction
Ilija Radosavovic, Bike Zhang, Baifeng Shi +5
We cast real-world humanoid control as a next token prediction problem, akin to predicting the next word in language. Our model is a causal transformer trained via autoregressive p…
Synthesizing Moving People with 3D Control
Boyi Li, Junming Chen, Jathushan Rajasegaran +3
In this paper, we present a diffusion model-based framework for animating people from a single image for a given target 3D motion sequence. Our approach has two core components: a)…
Humans in 4D: Reconstructing and Tracking Humans with Transformers
Shubham Goel, Georgios Pavlakos, Jathushan Rajasegaran +2
We present an approach to reconstruct humans and track them over time. At the core of our approach, we propose a fully "transformerized" version of a network for human mesh recover…
On the Benefits of 3D Pose and Tracking for Human Action Recognition
Jathushan Rajasegaran, Georgios Pavlakos, Angjoo Kanazawa +2
In this work we study the benefits of using tracking and 3D poses for action recognition. To achieve this, we take the Lagrangian view on analysing actions over a trajectory of hum…
Fully Online Meta-Learning Without Task Boundaries
Jathushan Rajasegaran, Chelsea Finn, Sergey Levine
While deep networks can learn complex functions such as classifiers, detectors, and trackers, many applications require models that continually adapt to changing input distribution…
Tracking People by Predicting 3D Appearance, Location & Pose
Jathushan Rajasegaran, Georgios Pavlakos, Angjoo Kanazawa +1
In this paper, we present an approach for tracking people in monocular videos, by predicting their future 3D representations. To achieve this, we first lift people to 3D from a sin…
Tracking People with 3D Representations
Jathushan Rajasegaran, Georgios Pavlakos, Angjoo Kanazawa +1
We present a novel approach for tracking multiple people in video. Unlike past approaches which employ 2D representations, we focus on using 3D representations of people, located i…
Meta-learning the Learning Trends Shared Across Tasks
Jathushan Rajasegaran, Salman Khan, Munawar Hayat +2
Meta-learning stands for 'learning to learn' such that generalization to new tasks is achieved. Among these methods, Gradient-based meta-learning algorithms are a specific sub-clas…
A Multi-modal Neural Embeddings Approach for Detecting Mobile Counterfeit Apps: A Case Study on Google Play Store
Naveen Karunanayake, Jathushan Rajasegaran, Ashanie Gunathillake +2
Counterfeit apps impersonate existing popular apps in attempts to misguide users to install them for various reasons such as collecting personal information or spreading malware. M…
Self-supervised Knowledge Distillation for Few-shot Learning
Jathushan Rajasegaran, Salman Khan, Munawar Hayat +2
Real-world contains an overwhelmingly large number of object classes, learning all of which at once is infeasible. Few shot learning is a promising learning paradigm due to its abi…
iTAML: An Incremental Task-Agnostic Meta-learning Approach
Jathushan Rajasegaran, Salman Khan, Munawar Hayat +2
Humans can continuously learn new knowledge as their experience grows. In contrast, previous learning in deep neural networks can quickly fade out when they are trained on a new ta…
Incremental Object Detection via Meta-Learning
K J Joseph, Jathushan Rajasegaran, Salman Khan +2
In a real-world setting, object instances from new classes can be continuously encountered by object detectors. When existing object detectors are applied to such scenarios, their…
TimeCaps: Capturing Time Series Data With Capsule Networks
Hirunima Jayasekara, Vinoj Jayasundara, Mohamed Athif +4
Capsule networks excel in understanding spatial relationships in 2D data for vision related tasks. Even though they are not designed to capture 1D temporal relationships, with Time…
An Adaptive Random Path Selection Approach for Incremental Learning
Jathushan Rajasegaran, Munawar Hayat, Salman Khan +3
In a conventional supervised learning setting, a machine learning model has access to examples of all object classes that are desired to be recognized during the inference stage. T…
DeepCaps: Going Deeper with Capsule Networks
Jathushan Rajasegaran, Vinoj Jayasundara, Sandaru Jayasekara +3
Capsule Network is a promising concept in deep learning, yet its true potential is not fully realized thus far, providing sub-par performance on several key benchmark datasets with…