1.6k citations · 1.8k across the 21 of their papers we have counts for
3 papers · 1 filter
MMMT-IF: A Challenging Multimodal Multi-Turn Instruction Following Benchmark
Elliot L. Epstein, Kaisheng Yao, Jing Li +2
Evaluating instruction following capabilities for multimodal, multi-turn dialogue is challenging. With potentially multiple instructions in the input model context, the task is tim…
Eureka: Evaluating and Understanding Large Foundation Models
Vidhisha Balachandran, Jingya Chen, Neel Joshi +6
Rigorous and reproducible evaluation is critical for assessing the state of the art and for guiding scientific advances in Artificial Intelligence. Evaluation is challenging in pra…
Improving Black-box Robustness with In-Context Rewriting
Kyle O'Brien, Nathan Ng, Isha Puri +5
Machine learning models for text classification often excel on in-distribution (ID) data but struggle with unseen out-of-distribution (OOD) inputs. Most techniques for improving OO…