7 papers
A Benchmark for Omni-Modal Reasoning in Long Videos
Mohammed Irfan Kurpath, Jaseel Muhammad Kaithakkodan, Jinxing Zhou +12
Long-form omni-modal video understanding requires integrating vision, speech, and ambient audio with coherent long-context reasoning. Existing video benchmarks often trade off temp…
MAviS: A Multimodal Conversational Assistant For Avian Species
Yevheniia Kryklyvets, Mohammed Irfan Kurpath, Sahal Shaji Mullappilly +5
Fine-grained understanding and species-specific multimodal question answering are vital for advancing biodiversity conservation and ecological monitoring. However, existing multimo…
MediX-R1: Open Ended Medical Reinforcement Learning
Sahal Shaji Mullappilly, Mohammed Irfan Kurpath, Omair Mohamed +5
We introduce MediX-R1, an open-ended Reinforcement Learning (RL) framework for medical multimodal large language models (MLLMs) that enables clinically grounded, free-form answers…
BiMediX2: Bio-Medical EXpert LMM for Diverse Medical Modalities
Sahal Shaji Mullappilly, Mohammed Irfan Kurpath, Sara Pieri +8
We introduce BiMediX2, a bilingual (Arabic-English) Bio-Medical EXpert Large Multimodal Model that supports text-based and image-based medical interactions. It enables multi-turn c…
XrayGPT: Chest Radiographs Summarization using Medical Vision-Language Models
Omkar Thawakar, Abdelrahman Shaker, Sahal Shaji Mullappilly +5
The latest breakthroughs in large vision-language models, such as Bard and GPT-4, have showcased extraordinary abilities in performing a wide range of tasks. Such models are traine…
LLMVoX: Autoregressive Streaming Text-to-Speech Model for Any LLM
Sambal Shikhar, Mohammed Irfan Kurpath, Sahal Shaji Mullappilly +5
Recent advancements in speech-to-speech dialogue systems leverage LLMs for multimodal interactions, yet they remain hindered by fine-tuning requirements, high computational overhea…