33 citations · 133 across the 57 of their papers we have counts for
49 papers · 1 filter
MediX-R1: Open Ended Medical Reinforcement Learning
Sahal Shaji Mullappilly, Mohammed Irfan Kurpath, Omair Mohamed +5
We introduce MediX-R1, an open-ended Reinforcement Learning (RL) framework for medical multimodal large language models (MLLMs) that enables clinically grounded, free-form answers…
GenZSL: Generative Zero-Shot Learning Via Inductive Variational Autoencoder
Shiming Chen, Dingjie Fu, Salman Khan +1
Remarkable progress in zero-shot learning (ZSL) has been achieved using generative models. However, existing generative ZSL methods merely generate (imagine) the visual features fr…
ThinkGeo: Evaluating Tool-Augmented Agents for Remote Sensing Tasks
Akashah Shabbir, Muhammad Akhtar Munir, Akshay Dudhane +6
Recent progress in large language models (LLMs) has enabled tool-augmented agents capable of solving complex real-world tasks through step-by-step reasoning. However, existing eval…
Deep Learning in Concealed Dense Prediction
Pancheng Zhao, Deng-Ping Fan, Shupeng Cheng +5
Deep learning is developing rapidly and handling common computer vision tasks well. It is time to pay attention to more complex vision tasks, as model size, knowledge, and reasonin…
DriveLMM-o1: A Step-by-Step Reasoning Dataset and Large Multimodal Model for Driving Scenario Understanding
Ayesha Ishaq, Jean Lahoud, Ketan More +10
While large multimodal models (LMMs) have demonstrated strong performance across various Visual Question Answering (VQA) tasks, certain challenges require complex multi-step reason…
Hierarchical Self-Supervised Adversarial Training for Robust Vision Models in Histopathology
Hashmat Shadab Malik, Shahina Kunhimon, Muzammal Naseer +2
Adversarial attacks pose significant challenges for vision models in critical fields like healthcare, where reliability is essential. Although adversarial training has been well st…
C-Drag: Chain-of-Thought Driven Motion Controller for Video Generation
Yuhao Li, Mirana Claire Angel, Salman Khan +4
Trajectory-based motion control has emerged as an intuitive and efficient approach for controllable video generation. However, the existing trajectory-based approaches are usually…
Time Travel: A Comprehensive Benchmark to Evaluate LMMs on Historical and Cultural Artifacts
Sara Ghaboura, Ketan More, Ritesh Thawkar +6
Understanding historical and cultural artifacts demands human expertise and advanced computational techniques, yet the process remains complex and time-intensive. While large multi…
AIN: The Arabic INclusive Large Multimodal Model
Ahmed Heakl, Sara Ghaboura, Omkar Thawkar +4
Amid the swift progress of large language models (LLMs) and their evolution into large multimodal models (LMMs), significant strides have been made in high-resource languages such…
Robust-LLaVA: On the Effectiveness of Large-Scale Robust Image Encoders for Multi-modal Large Language Models
Hashmat Shadab Malik, Fahad Shamshad, Muzammal Naseer +3
Multi-modal Large Language Models (MLLMs) excel in vision-language tasks but remain vulnerable to visual adversarial perturbations that can induce hallucinations, manipulate respon…
GeoPixel: Pixel Grounding Large Multimodal Model in Remote Sensing
Akashah Shabbir, Mohammed Zumri, Mohammed Bennamoun +2
Recent advances in large multimodal models (LMMs) have recognized fine-grained grounding as an imperative factor of visual understanding and dialogue. However, the benefits of such…
Discriminative Image Generation with Diffusion Models for Zero-Shot Learning
Dingjie Fu, Wenjin Hou, Shiming Chen +4
Generative Zero-Shot Learning (ZSL) methods synthesize class-related features based on predefined class semantic prototypes, showcasing superior performance. However, this feature…
UniMed-CLIP: Towards a Unified Image-Text Pretraining Paradigm for Diverse Medical Imaging Modalities
Muhammad Uzair Khattak, Shahina Kunhimon, Muzammal Naseer +2
Vision-Language Models (VLMs) trained via contrastive learning have achieved notable success in natural image tasks. However, their application in the medical domain remains limite…
EarthDial: Turning Multi-sensory Earth Observations to Interactive Dialogues
Sagar Soni, Akshay Dudhane, Hiyam Debary +8
Automated analysis of vast Earth observation data via interactive Vision-Language Models (VLMs) can unlock new opportunities for environmental monitoring, disaster response, and {r…
GEOBench-VLM: Benchmarking Vision-Language Models for Geospatial Tasks
Muhammad Sohail Danish, Muhammad Akhtar Munir, Syed Roshaan Ali Shah +5
While numerous recent benchmarks focus on evaluating generic Vision-Language Models (VLMs), they do not effectively address the specific challenges of geospatial applications. Gene…
VideoGLaMM: A Large Multimodal Model for Pixel-Level Visual Grounding in Videos
Shehan Munasinghe, Hanan Gani, Wenqi Zhu +4
Fine-grained alignment between videos and text is challenging due to complex spatial and temporal dynamics in videos. Existing video-based Large Multimodal Models (LMMs) handle bas…
CAMEL-Bench: A Comprehensive Arabic LMM Benchmark
Sara Ghaboura, Ahmed Heakl, Omkar Thawakar +7
Recent years have witnessed a significant interest in developing large multimodal models (LMMs) capable of performing various visual reasoning and understanding tasks. This has led…
How to Continually Adapt Text-to-Image Diffusion Models for Flexible Customization?
Jiahua Dong, Wenqi Liang, Hongliu Li +5
Custom diffusion models (CDMs) have attracted widespread attention due to their astonishing generative ability for personalized concepts. However, most existing CDMs unreasonably a…
AgriCLIP: Adapting CLIP for Agriculture and Livestock via Domain-Specialized Cross-Model Alignment
Umair Nawaz, Muhammad Awais, Hanan Gani +4
Capitalizing on vast amount of image-text data, large-scale vision-language pre-training has demonstrated remarkable zero-shot capabilities and has been utilized in several applica…
Open3DTrack: Towards Open-Vocabulary 3D Multi-Object Tracking
Ayesha Ishaq, Mohamed El Amine Boudjoghra, Jean Lahoud +4
3D multi-object tracking plays a critical role in autonomous driving by enabling the real-time monitoring and prediction of multiple objects' movements. Traditional 3D tracking sys…
CDChat: A Large Multimodal Model for Remote Sensing Change Description
Mubashir Noman, Noor Ahsan, Muzammal Naseer +4
Large multimodal models (LMMs) have shown encouraging performance in the natural image domain using visual instruction tuning. However, these LMMs struggle to describe the content…
BAPLe: Backdoor Attacks on Medical Foundational Models using Prompt Learning
Asif Hanif, Fahad Shamshad, Muhammad Awais +5
Medical foundation models are gaining prominence in the medical community for their ability to derive general representations from extensive collections of medical image-text pairs…
Open-Vocabulary Temporal Action Localization using Multimodal Guidance
Akshita Gupta, Aditya Arora, Sanath Narayan +3
Open-Vocabulary Temporal Action Localization (OVTAL) enables a model to recognize any desired action category in videos without the need to explicitly curate training data for all…
VideoGPT+: Integrating Image and Video Encoders for Enhanced Video Understanding
Muhammad Maaz, Hanoona Rasheed, Salman Khan +1
Building on the advances of language models, Large Multimodal Models (LMMs) have contributed significant improvements in video understanding. While the current video LMMs utilize a…