9 papers
A Technical Typology of AI Systems in Public Administration
Jonathan Rystrøm, Chris Schmitz, Nathan Davies +3
Research on artificial intelligence (AI) in the public sector often treats "AI" as a single category, neglecting technical distinctions between different AI systems. But these dist…
The Heterogeneous Safety Impacts of Benign Multilingual Fine-Tuning
Will Hawkins, Kaivalya Rawal, Jonathan Rystrøm +8
Fine-tuning a large language model is a ubiquitous method for enhancing its capability on a specific downstream task. However, prior work has shown that this increase in capability…
Grounding Text Embeddings in Stakeholder Associations
Jonathan Rystrøm, Sofie Burgos-Thorsen, Zihao Fu +3
Text embeddings are widely used to analyse large corpora of complex texts. However, it is unclear whether the embeddings capture the same semantic distances as the human experts us…
OxEnsemble: Fair Ensembles for Low-Data Classification
Jonathan Rystrøm, Zihao Fu, Chris Russell
We address the problem of fair classification in settings where data is scarce and unbalanced across demographic groups. Such low-data regimes are common in domains like medical im…
Agent Benchmarks Fail Public Sector Requirements
Jonathan Rystrøm, Chris Schmitz, Karolina Korgul +2
Deploying Large Language Model-based agents (LLM agents) in the public sector requires assuring that they meet the stringent legal, procedural, and structural requirements of publi…
MMTEB: Massive Multilingual Text Embedding Benchmark
Kenneth Enevoldsen, Isaac Chung, Imene Kerboua +83
Text embeddings are typically evaluated on a limited set of tasks, which are constrained by language, domain, and task diversity. To address these limitations and provide a more co…