20 citations · 56 across the 36 of their papers we have counts for
38 papers
The IOL-AI Challenge: An Open Challenge towards Advancing Linguistic Reasoning
Eduardo Sánchez, Rita Berrada, Dan-Mircea Mirea +8
Reasoning in LLMs is overwhelmingly studied in domains that provide a model with rules: mathematics and code. Linguistic puzzles invert this: the solver must first discover the sys…
CALIBER: Calibrating Confidence Before and After Reasoning in Language Models
Conor Finlay, Joshua Kurien, Saurabh Dash +2
Reasoning language models are increasingly asked not only to answer difficult questions, but also to estimate their likelihood of success. Existing methods typically elicit confide…
AI Exposure Scores: what they measure, what they miss, and what comes next
Campbell Lund, Thomas Euyang, Zanele Munyikwa +1
A set of exposure scores calculated in 2023 has become a central empirical input to the future of work debate. Produced by Eloundou et al. (2023) and referred to here as the GPTs a…
The Culture Funnel: You Can't Align What isn't in the Data
Ananya Sahu, Mehrnaz Mofakhami, Daniel D'Souza +3
Current cultural alignment approaches focus on inference-time interventions, assuming models already contain sufficient cultural knowledge. We argue modern LLM pipelines suffer fro…
CIRCLE: A Framework for Evaluating AI from a Real-World Lens
Reva Schwartz, Carina Westling, Morgan Briggs +12
This paper proposes CIRCLE, a six-stage, lifecycle-based framework to bridge the reality gap between model-centric performance metrics and AI's materialized outcomes in deployment.…
Tiny Aya: Bridging Scale and Multilingual Depth
Alejandro R. Salamanca, Diana Abagyan, Daniel D'souza +23
Tiny Aya redefines what a small multilingual language model can achieve. Trained on 70 languages and refined through region-aware posttraining, it delivers state-of-the-art in tran…