18 papers · 1 filter
BioVLM: Routing Prompts, Not Parameters, for Cross-Modality Generalization in Biomedical VLMs
Mainak Singha, Tanisha Gupta, Ankit Jha +3
Pretrained biomedical vision-language models (VLMs) such as BioMedCLIP perform well on average but often degrade on challenging modalities where inter-class margins are small and a…
bi-modal textual prompt learning for vision-language models in remote sensing
Pankhi Kashyap, Mainak Singha, Biplab Banerjee
Prompt learning (PL) has emerged as an effective strategy to adapt vision-language models (VLMs), such as CLIP, for downstream tasks under limited supervision. While PL has demonst…
MMLGNet: Cross-Modal Alignment of Remote Sensing Data using CLIP
Aditya Chaudhary, Sneha Barman, Mainak Singha +3
In this paper, we propose a novel multimodal framework, Multimodal Language-Guided Network (MMLGNet), to align heterogeneous remote sensing modalities like Hyperspectral Imaging (H…
SDHSI-Net: Learning Better Representations for Hyperspectral Images via Self-Distillation
Prachet Dev Singh, Shyamsundar Paramasivam, Sneha Barman +4
Hyperspectral image (HSI) classification presents unique challenges due to its high spectral dimensionality and limited labeled data. Traditional deep learning models often suffer…
Reconstruction Guided Few-shot Network For Remote Sensing Image Classification
Mohit Jaiswal, Naman Jain, Shivani Pathak +4
Few-shot remote sensing image classification is challenging due to limited labeled samples and high variability in land-cover types. We propose a reconstruction-guided few-shot net…
Label-Efficient Hyperspectral Image Classification via Spectral FiLM Modulation of Low-Level Pretrained Diffusion Features
Yuzhen Hu, Biplab Banerjee, Saurabh Prasad
Hyperspectral imaging (HSI) enables detailed land cover classification, yet low spatial resolution and sparse annotations pose significant challenges. We present a label-efficient…
FedMVP: Federated Multimodal Visual Prompt Tuning for Vision-Language Models
Mainak Singha, Subhankar Roy, Sarthak Mehrotra +4
In federated learning, textual prompt tuning adapts Vision-Language Models (e.g., CLIP) by tuning lightweight input tokens (or prompts) on local client data, while keeping network…
OSLoPrompt: Bridging Low-Supervision Challenges and Open-Set Domain Generalization in CLIP
Mohamad Hassan N C, Divyam Gupta, Mainak Singha +4
We introduce Low-Shot Open-Set Domain Generalization (LSOSDG), a novel paradigm unifying low-shot learning with open-set domain generalization (ODG). While prompt-based methods usi…
Foundation Models and Adaptive Feature Selection: A Synergistic Approach to Video Question Answering
Sai Bhargav Rongali, Mohamad Hassan N C, Ankit Jha +3
This paper tackles the intricate challenge of video question-answering (VideoQA). Despite notable progress, current methods fall short of effectively integrating questions with vid…
GraphVL: Graph-Enhanced Semantic Modeling via Vision-Language Models for Generalized Class Discovery
Bhupendra Solanki, Ashwin Nair, Mainak Singha +3
Generalized Category Discovery (GCD) aims to cluster unlabeled images into known and novel categories using labeled images from known classes. To address the challenge of transferr…
COSMo: CLIP Talks on Open-Set Multi-Target Domain Adaptation
Munish Monga, Sachin Kumar Giroh, Ankit Jha +3
Multi-Target Domain Adaptation (MTDA) entails learning domain-invariant information from a single source domain and applying it to multiple unlabeled target domains. Yet, existing…
Incremental Open-set Domain Adaptation
Sayan Rakshit, Hmrishav Bandyopadhyay, Nibaran Das +1
Catastrophic forgetting makes neural network models unstable when learning visual domains consecutively. The neural network model drifts to catastrophic forgetting-induced low perf…
Elevating All Zero-Shot Sketch-Based Image Retrieval Through Multimodal Prompt Learning
Mainak Singha, Ankit Jha, Divyam Gupta +2
We address the challenges inherent in sketch-based image retrieval (SBIR) across various settings, including zero-shot SBIR, generalized zero-shot SBIR, and fine-grained zero-shot…
CDAD-Net: Bridging Domain Gaps in Generalized Category Discovery
Sai Bhargav Rongali, Sarthak Mehrotra, Ankit Jha +5
In Generalized Category Discovery (GCD), we cluster unlabeled samples of known and novel classes, leveraging a training dataset of known classes. A salient challenge arises due to…
Unknown Prompt, the only Lacuna: Unveiling CLIP's Potential for Open Domain Generalization
Mainak Singha, Ankit Jha, Shirsha Bose +3
We delve into Open Domain Generalization (ODG), marked by domain and category shifts between training's labeled source and testing's unlabeled target domains. Existing solutions to…
C-SAW: Self-Supervised Prompt Learning for Image Generalization in Remote Sensing
Avigyan Bhattacharya, Mainak Singha, Ankit Jha +1
We focus on domain and class generalization problems in analyzing optical remote sensing images, using the large-scale pre-trained vision-language model (VLM), CLIP. While contrast…
Learning Class and Domain Augmentations for Single-Source Open-Domain Generalization
Prathmesh Bele, Valay Bundele, Avigyan Bhattacharya +3
Single-source open-domain generalization (SS-ODG) addresses the challenge of labeled source domains with supervision during training and unlabeled novel target domains during testi…
Domain Adaptive Few-Shot Open-Set Learning
Debabrata Pal, Deeptej More, Sai Bhargav +3
Few-shot learning has made impressive strides in addressing the crucial challenges of recognizing unknown samples from novel classes in target query sets and managing visual shifts…