52 citations · 143 across the 25 of their papers we have counts for
21 papers · 1 filter
Beyond Similarity: Foundation Models as an Efficient Backbone for Training-Free Composed Video Retrieval
Dmitry Demidov, Muhammad Zaigham Zaheer, Omkar Thawakar +2
Composed video retrieval (CoVR) searches a gallery for the target video that realizes a natural-language modification of a source clip. However, at gallery scale, this creates a fu…
Perception Before Supervision: Self-Contained Visual Distillation from Counterfactual Blind Spots
Shravan Venkatraman, Omkar Thawakar, Ritesh Thawkar +2
Self-improvement for multimodal large language models (MLLMs) is typically driven by reward-based methods that provide only coarse scalar feedback. Distillation offers a richer alt…
Ask, Solve, Generate: Self-Evolving Unified Multimodal Understanding and Generation via Self-Consistency Rewards
Ritesh Thawkar, Shravan Venkatraman, Omkar Thawakar +5
Most unified large multimodal models (LMMs) that support both visual understanding and image generation still rely on curated post-training supervision, such as human annotations,…
Not All Modalities Are Equal: Instruction-Aware Gating for Multimodal Videos
Bonan Ding, Umair Nawaz, Ufaq Khan +5
Pre-trained video large language models excel at visual reasoning. However, they struggle when videos arrive with auxiliary streams, such as audio, depth map, or dense temporal evi…
WorldCache: Content-Aware Caching for Accelerated Video World Models
Umair Nawaz, Ahmed Heakl, Ufaq Khan +3
Diffusion Transformers (DiTs) power high-fidelity video world models but remain computationally expensive due to sequential denoising and costly spatio-temporal attention. Training…
Mobile-O: Unified Multimodal Understanding and Generation on Mobile Device
Abdelrahman Shaker, Ahmed Heakl, Jaseel Muhammad +8
Unified multimodal models can both understand and generate visual content within a single architecture. Existing models, however, remain data-hungry and too heavy for deployment on…
Diversity Has Always Been There in Your Visual Autoregressive Models
Tong Wang, Guanyu Yang, Nian Liu +6
Visual Autoregressive (VAR) models have recently garnered significant attention for their innovative next-scale prediction paradigm, offering notable advantages in both inference e…
EvoLMM: Self-Evolving Large Multimodal Models with Continuous Rewards
Omkar Thawakar, Shravan Venkatraman, Ritesh Thawkar +5
Recent advances in large multimodal models (LMMs) have enabled impressive reasoning and perception abilities, yet most existing training pipelines still depend on human-curated dat…
MATRIX: Multimodal Agent Tuning for Robust Tool-Use Reasoning
Tajamul Ashraf, Umair Nawaz, Abdelrahman M. Shaker +4
Vision language models (VLMs) are increasingly deployed as controllers with access to external tools for complex reasoning and decision-making, yet their effectiveness remains limi…
VideoMolmo: Spatio-Temporal Grounding Meets Pointing
Ghazi Shazan Ahmad, Ahmed Heakl, Hanan Gani +4
Spatio-temporal localization is vital for precise interactions across diverse domains, from biological research to autonomous navigation and interactive interfaces. Current video-b…
VideoMathQA: Benchmarking Mathematical Reasoning via Multimodal Understanding in Videos
Hanoona Rasheed, Abdelrahman Shaker, Anqi Tang +4
Mathematical reasoning in real-world video settings presents a fundamentally different challenge than in static images or text. It requires interpreting fine-grained visual informa…
Mobile-VideoGPT: Fast and Accurate Model for Mobile Video Understanding
Abdelrahman Shaker, Muhammad Maaz, Chenhui Gou +3
Video understanding models often struggle with high computational requirements, extensive parameter counts, and slow inference speed, making them inefficient for practical use. To…
All Languages Matter: Evaluating LMMs on Culturally Diverse 100 Languages
Ashmal Vayani, Dinura Dissanayake, Hasindri Watawana +66
Existing Large Multimodal Models (LMMs) generally focus on only a few regions and languages. As LMMs continue to improve, it is increasingly important to ensure they understand cul…
GroupMamba: Efficient Group-Based Visual State Space Model
Abdelrahman Shaker, Syed Talal Wasim, Salman Khan +2
State-space models (SSMs) have recently shown promise in capturing long-range dependencies with subquadratic computational complexity, making them attractive for various applicatio…
Efficient Video Object Segmentation via Modulated Cross-Attention Memory
Abdelrahman Shaker, Syed Talal Wasim, Martin Danelljan +3
Recently, transformer-based approaches have shown promising results for semi-supervised video object segmentation. However, these approaches typically struggle on long videos due t…
GLaMM: Pixel Grounding Large Multimodal Model
Hanoona Rasheed, Muhammad Maaz, Sahal Shaji Mullappilly +7
Large Multimodal Models (LMMs) extend Large Language Models to the vision domain. Initial LMMs used holistic images and text prompts to generate ungrounded textual responses. Recen…
XrayGPT: Chest Radiographs Summarization using Medical Vision-Language Models
Omkar Thawakar, Abdelrahman Shaker, Sahal Shaji Mullappilly +5
The latest breakthroughs in large vision-language models, such as Bard and GPT-4, have showcased extraordinary abilities in performing a wide range of tasks. Such models are traine…
SwiftFormer: Efficient Additive Attention for Transformer-based Real-time Mobile Vision Applications
Abdelrahman Shaker, Muhammad Maaz, Hanoona Rasheed +3
Self-attention has become a defacto choice for capturing global context in various vision applications. However, its quadratic computational complexity with respect to image resolu…
UNETR++: Delving into Efficient and Accurate 3D Medical Image Segmentation
Abdelrahman Shaker, Muhammad Maaz, Hanoona Rasheed +3
Owing to the success of transformer models, recent works study their applicability in 3D medical segmentation tasks. Within the transformer models, the self-attention mechanism is…
EdgeNeXt: Efficiently Amalgamated CNN-Transformer Architecture for Mobile Vision Applications
Muhammad Maaz, Abdelrahman Shaker, Hisham Cholakkal +4
In the pursuit of achieving ever-increasing accuracy, large and complex neural networks are usually developed. Such models demand high computational resources and therefore cannot…
INSTA-YOLO: Real-Time Instance Segmentation
Eslam Mohamed, Abdelrahman Shaker, Ahmad El-Sallab +1
Instance segmentation has gained recently huge attention in various computer vision applications. It aims at providing different IDs to different object of the scene, even if they…