565 citations · 908 across the 35 of their papers we have counts for
38 papers
Seven simple steps for log analysis in AI systems
Magda Dubois, Ekin Zorer, Maia Hamin +17
AI systems produce large volumes of logs as they interact with tools and users. Analysing these logs can help understand model capabilities, propensities, and behaviours, or assess…
Relative Drawing Identification Complexity is Invariant to Modality in Vision-Language Models
Diogo Freitas, Brigt Håvardstun, Cèsar Ferri +3
Large language models have become multimodal, and many of them are said to integrate their modalities using common representations. If this were true, a drawing of a car as an imag…
Open Problems in Machine Unlearning for AI Safety
Fazl Barez, Tingchen Fu, Ameya Prabhu +16
As AI systems become more capable, widely deployed, and increasingly autonomous in critical areas such as cybersecurity, biological research, and healthcare, ensuring their safety…
Learning Alternative Ways of Performing a Task
David Nieves, María José Ramírez-Quintana, Carlos Monserrat +2
A common way of learning to perform a task is to observe how it is carried out by experts. However, it is well known that for most tasks there is no unique way to perform them. Thi…
Foundational Challenges in Assuring Alignment and Safety of Large Language Models
Usman Anwar, Abulhair Saparov, Javier Rando +39
This work identifies 18 foundational challenges in assuring the alignment and safety of large language models (LLMs). These challenges are organized into three different categories…
Evaluating General-Purpose AI with Psychometrics
Xiting Wang, Liming Jiang, Jose Hernandez-Orallo +4
Comprehensive and accurate evaluation of general-purpose AI systems such as large language models allows for effective mitigation of their risks and deepened understanding of their…