36 citations · 40 across the 3 of their papers we have counts for
6 papers
Measuring what Matters: Construct Validity in Large Language Model Benchmarks
Andrew M. Bean, Ryan Othniel Kearns, Angelika Romanou +39
Evaluating large language models (LLMs) is crucial for both assessing their capabilities and identifying safety or robustness issues prior to deployment. Reliably measuring abstrac…
Apertus: Democratizing Open and Compliant LLMs for Global Language Environments
Project Apertus, Alejandro Hernández-Cano, Alexander Hägele +100
We present Apertus, a fully open suite of large language models (LLMs) designed to address two systemic shortcomings in today's open model ecosystem: data compliance and multilingu…
Challenges for AI in Multimodal STEM Assessments: a Human-AI Comparison
Aymeric de Chillaz, Anna Sotnikova, Patrick Jermann +1
Generative AI systems have rapidly advanced, with multimodal input capabilities enabling reasoning beyond text-based tasks. In education, these advancements could influence assessm…
INCLUDE: Evaluating Multilingual Language Understanding with Regional Knowledge
Angelika Romanou, Negar Foroutan, Anna Sotnikova +56
The performance differential of large language models (LLM) between languages hinders their effective deployment in many regions, inhibiting the potential economic and societal val…
Could ChatGPT get an Engineering Degree? Evaluating Higher Education Vulnerability to AI Assistants
Beatriz Borges, Negar Foroutan, Deniz Bayazit +87
AI assistants are being increasingly used by students enrolled in higher education institutions. While these tools provide opportunities for improved teaching and education, they a…
"Flex Tape Can't Fix That": Bias and Misinformation in Edited Language Models
Karina Halevy, Anna Sotnikova, Badr AlKhamissi +2
Model editing has emerged as a cost-effective strategy to update knowledge stored in language models. However, model editing can have unintended consequences after edits are applie…