74 citations · 139 across the 13 of their papers we have counts for
14 papers
Imagine How To Change: Explicit Procedure Modeling for Change Captioning
Jiayang Sun, Zixin Guo, Min Cao +2
Change captioning generates descriptions that explicitly describe the differences between two visually similar images. Existing methods operate on static image pairs, thus ignoring…
MIRA: A Novel Framework for Fusing Modalities in Medical RAG
Jinhong Wang, Tajamul Ashraf, Zongyan Han +2
Multimodal Large Language Models (MLLMs) have significantly advanced AI-assisted medical diagnosis, but they often generate factually inconsistent responses that deviate from estab…
ARB: A Comprehensive Arabic Multimodal Reasoning Benchmark
Sara Ghaboura, Ketan More, Wafa Alghallabi +5
As Large Multimodal Models (LMMs) become more capable, there is growing interest in evaluating their reasoning processes alongside their final outputs. However, most benchmarks rem…
OpenSeg-R: Improving Open-Vocabulary Segmentation via Step-by-Step Visual Reasoning
Zongyan Han, Jiale Cao, Shuo Chen +3
Open-Vocabulary Segmentation (OVS) has drawn increasing attention for its capacity to generalize segmentation beyond predefined categories. However, existing methods typically pred…
All Languages Matter: Evaluating LMMs on Culturally Diverse 100 Languages
Ashmal Vayani, Dinura Dissanayake, Hasindri Watawana +66
Existing Large Multimodal Models (LMMs) generally focus on only a few regions and languages. As LMMs continue to improve, it is increasingly important to ensure they understand cul…
CAMEL-Bench: A Comprehensive Arabic LMM Benchmark
Sara Ghaboura, Ahmed Heakl, Omkar Thawakar +7
Recent years have witnessed a significant interest in developing large multimodal models (LMMs) capable of performing various visual reasoning and understanding tasks. This has led…
Bilateral Reference for High-Resolution Dichotomous Image Segmentation
Peng Zheng, Dehong Gao, Deng-Ping Fan +4
We introduce a novel bilateral reference framework (BiRefNet) for high-resolution dichotomous image segmentation (DIS). It comprises two essential components: the localization modu…
PiTL: Cross-modal Retrieval with Weakly-supervised Vision-language Pre-training via Prompting
Zixin Guo, Tzu-Jui Julius Wang, Selen Pehlivan +2
Vision-language (VL) Pre-training (VLP) has shown to well generalize VL models over a wide range of VL downstream tasks, especially for cross-modal retrieval. However, it hinges on…
XrayGPT: Chest Radiographs Summarization using Medical Vision-Language Models
Omkar Thawakar, Abdelrahman Shaker, Sahal Shaji Mullappilly +5
The latest breakthroughs in large vision-language models, such as Bard and GPT-4, have showcased extraordinary abilities in performing a wide range of tasks. Such models are traine…
Video Instance Segmentation in an Open-World
Omkar Thawakar, Sanath Narayan, Hisham Cholakkal +5
Existing video instance segmentation (VIS) approaches generally follow a closed-world assumption, where only seen category instances are identified and spatio-temporally segmented…
Cross-modulated Few-shot Image Generation for Colorectal Tissue Classification
Amandeep Kumar, Ankan kumar Bhunia, Sanath Narayan +4
In this work, we propose a few-shot colorectal tissue image generation method for addressing the scarcity of histopathological training data for rare cancer tissues. Our few-shot g…
3D Mitochondria Instance Segmentation with Spatio-Temporal Transformers
Omkar Thawakar, Rao Muhammad Anwer, Jorma Laaksonen +3
Accurate 3D mitochondria instance segmentation in electron microscopy (EM) is a challenging problem and serves as a prerequisite to empirically analyze their distributions and morp…
Person Image Synthesis via Denoising Diffusion Model
Ankan Kumar Bhunia, Salman Khan, Hisham Cholakkal +4
The pose-guided person image generation task requires synthesizing photorealistic images of humans in arbitrary poses. The existing approaches use generative adversarial networks t…
Learning by Hallucinating: Vision-Language Pre-training with Weak Supervision
Tzu-Jui Julius Wang, Jorma Laaksonen, Tomas Langer +2
Weakly-supervised vision-language (V-L) pre-training (W-VLP) aims at learning cross-modal alignment with little or no paired data, such as aligned images and captions. Recent W-VLP…