4 citations · 5 across the 12 of their papers we have counts for
12 papers
Attending to Multimodal Generation One Token at a Time
Varun Gupta, Vineet Gandhi, Makarand Tapaswi
Multimodal large language models (MLLMs) generate responses autoregressively, integrating visual and linguistic information in an evolving context. Prior work on interpretability h…
LiteEmbed: Adapting CLIP to Rare Classes
Aishwarya Agarwal, Srikrishna Karanam, Vineet Gandhi
Large-scale vision-language models such as CLIP achieve strong zero-shot recognition but struggle with classes that are rarely seen during pretraining, including newly emerging ent…
Concept Regions Matter: Benchmarking CLIP with a New Cluster-Importance Approach
Aishwarya Agarwal, Srikrishna Karanam, Vineet Gandhi
Contrastive vision-language models (VLMs) such as CLIP achieve strong zero-shot recognition yet remain vulnerable to spurious correlations, particularly background over-reliance. W…
Simplifying Knowledge Transfer in Pretrained Models
Siddharth Jain, Shyamgopal Karthik, Vineet Gandhi
Pretrained models are ubiquitous in the current deep learning landscape, offering strong results on a broad range of tasks. Recent works have shown that models differing in various…
Investigating Mechanisms for In-Context Vision Language Binding
Darshana Saravanan, Makarand Tapaswi, Vineet Gandhi
To understand a prompt, Vision-Language models (VLMs) must perceive the image, comprehend the text, and build associations within and across both modalities. For instance, given an…
EditIQ: Automated Cinematic Editing of Static Wide-Angle Videos via Dialogue Interpretation and Saliency Cues
Rohit Girmaji, Bhav Beri, Ramanathan Subramanian +1
We present EditIQ, a completely automated framework for cinematically editing scenes captured via a stationary, large field-of-view and high-resolution camera. From the static came…
Minimalistic Video Saliency Prediction via Efficient Decoder & Spatio Temporal Action Cues
Rohit Girmaji, Siddharth Jain, Bhav Beri +2
This paper introduces ViNet-S, a 36MB model based on the ViNet architecture with a U-Net design, featuring a lightweight decoder that significantly reduces model size and parameter…
TIDE: Training Locally Interpretable Domain Generalization Models Enables Test-time Correction
Aishwarya Agarwal, Srikrishna Karanam, Vineet Gandhi
We consider the problem of single-source domain generalization. Existing methods typically rely on extensive augmentations to synthetically cover diverse domains during training. H…
IdentifyMe: A Challenging Long-Context Mention Resolution Benchmark for LLMs
Kawshik Manikantan, Makarand Tapaswi, Vineet Gandhi +1
Recent evaluations of LLMs on coreference resolution have revealed that traditional output formats and evaluation metrics do not fully capture the models' referential understanding…
VELOCITI: Benchmarking Video-Language Compositional Reasoning with Strict Entailment
Darshana Saravanan, Varun Gupta, Darshan Singh +3
A fundamental aspect of compositional reasoning in a video is associating people and their actions across time. Recent years have seen great progress in general-purpose vision or v…
Pseudo-labelling meets Label Smoothing for Noisy Partial Label Learning
Darshana Saravanan, Naresh Manwani, Vineet Gandhi
We motivate weakly supervised learning as an effective learning paradigm for problems where curating perfectly annotated datasets is expensive and may require domain expertise such…
Real Time GAZED: Online Shot Selection and Editing of Virtual Cameras from Wide-Angle Monocular Video Recordings
Sudheer Achary, Rohit Girmaji, Adhiraj Anil Deshmukh +1
Eliminating time-consuming post-production processes and delivering high-quality videos in today's fast-paced digital landscape are the key advantages of real-time approaches. To a…