9 papers
Almieyar-Oryx-BloomBench: A Bilingual Multimodal Benchmark for Cognitively Informed Evaluation of Vision-Language Models
Mohammad Mahdi Abootorabi, Omid Ghahroodi, Anas Madkoor +4
Despite the rapid progress of Vision-Language Models (VLMs), the field lacks benchmarks that rigorously diagnose their true reasoning abilities and chart meaningful progress toward…
The Judge Who Never Admits: Hidden Shortcuts in LLM-based Evaluation
Arash Marioriyad, Omid Ghahroodi, Ehsaneddin Asgari +2
Large language models (LLMs) are increasingly used as automatic judges to evaluate system outputs in tasks such as reasoning, question answering, and creative writing. A faithful j…
Limits and Gains of Test-Time Scaling in Vision-Language Reasoning
Mohammadjavad Ahmadpour, Amirmahdi Meighani, Payam Taebi +3
Test-time scaling (TTS) has emerged as a powerful paradigm for improving the reasoning ability of Large Language Models (LLMs) by allocating additional computation at inference, ye…
ADAM: A Diverse Archive of Mankind for Evaluating and Enhancing LLMs in Biographical Reasoning
Jasin Cekinmez, Omid Ghahroodi, Saad Fowad Chandle +2
We introduce ADAM (A Diverse Archive of Mankind), a framework for evaluating and improving multimodal large language models (MLLMs) in biographical reasoning. To the best of our kn…
MEENA (PersianMMMU): Multimodal-Multilingual Educational Exams for N-level Assessment
Omid Ghahroodi, Arshia Hemmat, Marzia Nouri +8
Recent advancements in large vision-language models (VLMs) have primarily focused on English, with limited attention given to other languages. To address this gap, we introduce MEE…
Ask in Any Modality: A Comprehensive Survey on Multimodal Retrieval-Augmented Generation
Mohammad Mahdi Abootorabi, Amirhosein Zobeiri, Mahdi Dehghani +5
Large Language Models (LLMs) suffer from hallucinations and outdated knowledge due to their reliance on static training data. Retrieval-Augmented Generation (RAG) mitigates these i…