4 citations · 4 across the 6 of their papers we have counts for
5 papers · 1 filter
HumaniBench: A Human-Centric Framework for Large Multimodal Models Evaluation
Shaina Raza, Aravind Narayanan, Vahid Reza Khazaie +6
Although recent large multimodal models (LMMs) show impressive progress on vision language tasks, their alignment with human centered (HC) principles such as fairness, ethics, incl…
BBQ-V: Benchmarking Visual Stereotype Bias in Large Multimodal Models
Vishal Narnaware, Ashmal Vayani, Rohit Gupta +2
Stereotype biases in Large Multimodal Models (LMMs) perpetuate harmful societal prejudices, undermining the fairness and equity of AI applications. As LMMs grow increasingly influe…
GAEA: A Geolocation Aware Conversational Assistant
Ron Campos, Ashmal Vayani, Parth Parag Kulkarni +4
Image geolocalization, in which an AI model traditionally predicts the precise GPS coordinates of an image, is a challenging task with many downstream applications. However, the us…
All Languages Matter: Evaluating LMMs on Culturally Diverse 100 Languages
Ashmal Vayani, Dinura Dissanayake, Hasindri Watawana +66
Existing Large Multimodal Models (LMMs) generally focus on only a few regions and languages. As LMMs continue to improve, it is increasingly important to ensure they understand cul…
VURF: A General-purpose Reasoning and Self-refinement Framework for Video Understanding
Ahmad Mahmood, Ashmal Vayani, Muzammal Naseer +2
Recent studies have demonstrated the effectiveness of Large Language Models (LLMs) as reasoning modules that can deconstruct complex tasks into more manageable sub-tasks, particula…