3 citations · 9 across the 11 of their papers we have counts for
4 papers · 1 filter
Unearthing Skill-Level Insights for Understanding Trade-Offs of Foundation Models
Mazda Moayeri, Vidhisha Balachandran, Varun Chandrasekaran +6
With models getting stronger, evaluations have grown more complex, testing multiple skills in one benchmark and even in the same instance at once. However, skill-wise performance i…
BenchAgents: Multi-Agent Systems for Structured Benchmark Creation
Natasha Butt, Varun Chandrasekaran, Neel Joshi +2
Evaluation insights are limited by the availability of high-quality benchmarks. As models evolve, there is a need to create benchmarks that can measure progress on new and complex…
Eureka: Evaluating and Understanding Large Foundation Models
Vidhisha Balachandran, Jingya Chen, Neel Joshi +6
Rigorous and reproducible evaluation is critical for assessing the state of the art and for guiding scientific advances in Artificial Intelligence. Evaluation is challenging in pra…
Is A Picture Worth A Thousand Words? Delving Into Spatial Reasoning for Vision Language Models
Jiayu Wang, Yifei Ming, Zhenmei Shi +4
Large language models (LLMs) and vision-language models (VLMs) have demonstrated remarkable performance across a wide range of tasks and domains. Despite this promise, spatial unde…