4 citations · 4 across the 6 of their papers we have counts for
9 papers
Reject, Remember, Retry: Why Multi-Turn Attackers Should Remember Their Mistakes
Yuqing Huang, Jack Ji, Jithin Pradeep +3
Existing multi-turn jailbreak attackers condition on the same conversation history as the target. We introduce (Reject, Remember, Retry), a black-box multi-LLM framework in w…
When Judges Agree and Are Wrong: Consistency-Accuracy Dissociation as a Blind Spot for Label-Free Evaluation
Xiang Zhang, Lovedeep Gondara, Jekaterina Novikova
Label-free evaluation methods, such as self-consistency filtering, majority-vote aggregation, and agreement-based reliability scoring, rely on an implicit assumption that consisten…
The Format is the Filter: Scoring Format Fixes What an LLM Judge Can Express
Lovedeep Gondara, Jekaterina Novikova, Sagarika Suresh +2
When an LLM judge filters instruction-tuning data, its scoring format (binary verdict, 1--5 rating, 0--100 score, or comparison) is rarely justified or ablated. Holding the candida…
Supervised Learning Objectives for High-Dimensional Treatment Effects
Nikita Dhawan, Arnav Paruthi, Andrew Kim +3
Predicting the effect of interventions with many possible variations, e.g., therapeutic content that affects mental health outcomes or an earnings call transcript that drives movem…
AFRICAPTION: Establishing a New Paradigm for Image Captioning in African Languages
Mardiyyah Oduwole, Prince Mireku, Fatimo Adebanjo +3
Multimodal AI research has overwhelmingly focused on high-resource languages, hindering the democratization of advancements in the field. To address this, we present AfriCaption, a…
Consistency in Language Models: Current Landscape, Challenges, and Future Directions
Jekaterina Novikova, Carol Anderson, Borhane Blili-Hamelin +2
The hallmark of effective language use lies in consistency: expressing similar meanings in similar contexts and avoiding contradictions. While human communication naturally demonst…
Kaleidoscope: In-language Exams for Massively Multilingual Vision Evaluation
Israfel Salazar, Manuel Fernández Burda, Shayekh Bin Islam +42
The evaluation of vision-language models (VLMs) has mainly relied on English-language benchmarks, leaving significant gaps in both multilingual and multicultural coverage. While mu…
INCLUDE: Evaluating Multilingual Language Understanding with Regional Knowledge
Angelika Romanou, Negar Foroutan, Anna Sotnikova +56
The performance differential of large language models (LLM) between languages hinders their effective deployment in many regions, inhibiting the potential economic and societal val…
Out of the Ordinary: Spectrally Adapting Regression for Covariate Shift
Benjamin Eyre, Elliot Creager, David Madras +2
Designing deep neural network classifiers that perform robustly on distributions differing from the available training data is an active area of machine learning research. However,…