54 citations · 91 across the 16 of their papers we have counts for
16 papers · 1 filter
PISCES: Annotation-free Text-to-Video Post-Training via Optimal Transport-Aligned Rewards
Minh-Quan Le, Gaurav Mittal, Cheng Zhao +3
Text-to-video (T2V) generation aims to synthesize videos with high visual quality and temporal consistency that are semantically aligned with input text. Reward-based post-training…
Tracking-Guided 4D Generation: Foundation-Tracker Motion Priors for 3D Model Animation
Su Sun, Cheng Zhao, Himangi Mittal +4
Generating dynamic 4D objects from sparse inputs is difficult because it demands joint preservation of appearance and motion coherence across views and time while suppressing artif…
Scenes as Tokens: Multi-Scale Normal Distributions Transform Tokenizer for General 3D Vision-Language Understanding
Yutao Tang, Cheng Zhao, Gaurav Mittal +4
Recent advances in 3D vision-language models (VLMs) highlight a strong potential for 3D scene understanding and reasoning. However, effectively tokenizing 3D scenes into holistic s…
DeCafNet: Delegate and Conquer for Efficient Temporal Grounding in Long Videos
Zijia Lu, A S M Iftekhar, Gaurav Mittal +6
Long Video Temporal Grounding (LVTG) aims at identifying specific moments within lengthy videos based on user-provided text queries for effective content retrieval. The approach ta…
Hummingbird: High Fidelity Image Generation via Multimodal Context Alignment
Minh-Quan Le, Gaurav Mittal, Tianjian Meng +5
While diffusion models are powerful in generating high-quality, diverse synthetic data for object-centric tasks, existing methods struggle with scene-aware tasks such as Visual Que…
LoSA: Long-Short-range Adapter for Scaling End-to-End Temporal Action Localization
Akshita Gupta, Gaurav Mittal, Ahmed Magooda +3
Temporal Action Localization (TAL) involves localizing and classifying action snippets in an untrimmed video. The emergence of large video foundation models has led RGB-only video…
Rethinking Multimodal Content Moderation from an Asymmetric Angle with Mixed-modality
Jialin Yuan, Ye Yu, Gaurav Mittal +3
There is a rapidly growing need for multimodal content moderation (CM) as more and more content on social media is multimodal in nature. Existing unimodal CM systems may fail to ca…
BATMAN: Bilateral Attention Transformer in Motion-Appearance Neighboring Space for Video Object Segmentation
Ye Yu, Jialin Yuan, Gaurav Mittal +2
Video Object Segmentation (VOS) is fundamental to video understanding. Transformer-based methods show significant performance improvement on semi-supervised VOS. However, existing…
GateHUB: Gated History Unit with Background Suppression for Online Action Detection
Junwen Chen, Gaurav Mittal, Ye Yu +2
Online action detection is the task of predicting the action as soon as it happens in a streaming video. A major challenge is that the model does not have access to the future and…
MUSE: Feature Self-Distillation with Mutual Information and Self-Information
Yu Gong, Ye Yu, Gaurav Mittal +2
We present a novel information-theoretic approach to introduce dependency among features of a deep convolutional neural network (CNN). The core idea of our proposed method, called…
Unsupervised Few-Shot Action Recognition via Action-Appearance Aligned Meta-Adaptation
Jay Patravali, Gaurav Mittal, Ye Yu +2
We present MetaUVFS as the first Unsupervised Meta-learning algorithm for Video Few-Shot action recognition. MetaUVFS leverages over 550K unlabeled videos to train a two-stream 2D…
HyperSTAR: Task-Aware Hyperparameters for Deep Networks
Gaurav Mittal, Chang Liu, Nikolaos Karianakis +3
While deep neural networks excel in solving visual recognition tasks, they require significant effort to find hyperparameters that make them work optimally. Hyperparameter Optimiza…
Animating Face using Disentangled Audio Representations
Gaurav Mittal, Baoyuan Wang
All previous methods for audio-driven talking head generation assume the input audio to be clean with a neutral tone. As we show empirically, one can easily break these systems by…
Interactive Image Generation Using Scene Graphs
Gaurav Mittal, Shubham Agrawal, Anuva Agarwal +2
Recent years have witnessed some exciting developments in the domain of generating images from scene-based text descriptions. These approaches have primarily focused on generating…
Attentive Semantic Video Generation using Captions
Tanya Marwah, Gaurav Mittal, Vineeth N. Balasubramanian
This paper proposes a network architecture to perform variable length semantic video generation using captions. We adopt a new perspective towards video generation where we allow t…
Sync-DRAW: Automatic Video Generation using Deep Recurrent Attentive Architectures
Gaurav Mittal, Tanya Marwah, Vineeth N. Balasubramanian
This paper introduces a novel approach for generating videos called Synchronized Deep Recurrent Attentive Writer (Sync-DRAW). Sync-DRAW can also perform text-to-video generation wh…