4 papers
The Unwritten Benchmark: A New Challenge for Multimodal Machine Learning in Abstract Perceptual Reasoning
Garima Arya Yadav, Nilay Yilmaz, Yezhou Yang
Current multimodal models have demonstrated remarkable proficiency in recognizing static visual and auditory content. However, their capacity for abstract perceptual reasoning, inf…
SciReC: Diagnostic Evaluation of Multimodal, Multi-Turn Relational Reasoning with Adaptive Interaction
Nilay Yilmaz, Naga Sai Abhiram Kusumba, Stella Wenxing Liu +1
Relational reasoning requires the process of perceptual understanding, comparing, and integrating the underlying relationships between concepts. This ability consists of multiple c…
MentalBlackboard: Evaluating Spatial Visualization via Mathematical Transformations
Nilay Yilmaz, Maitreya Patel, Naga Sai Abhiram Kusumba +2
Spatial visualization is the mental ability to imagine, transform, and manipulate the spatial characteristics of objects and actions. This intelligence is a part of human cognition…
VOILA: Evaluation of MLLMs For Perceptual Understanding and Analogical Reasoning
Nilay Yilmaz, Maitreya Patel, Yiran Lawrence Luo +4
Multimodal Large Language Models (MLLMs) have become a powerful tool for integrating visual and textual information. Despite their exceptional performance on visual understanding b…