7 papers
Large Language Model Selection with Limited Annotations
Yavuz Durmazkeser, Patrik Okanovic, Andreas Kirsch +2
Choosing a Large Language Model (LLM) for a given task requires comparing many strong candidates, yet standard evaluation relies on costly annotations over fixed evaluation sets. T…
Active Model Selection for Large Language Models
Yavuz Durmazkeser, Patrik Okanovic, Andreas Kirsch +2
We introduce LLM SELECTOR, the first framework for active model selection of Large Language Models (LLMs). Unlike prior evaluation and benchmarking approaches that rely on fully an…
Evaluating the Limits of Large Language Models in Multilingual Legal Reasoning
Antreas Ioannou, Andreas Shiamishis, Nora Hollenstein +1
In an era dominated by Large Language Models (LLMs), understanding their capabilities and limitations, especially in high-stakes fields like law, is crucial. While LLMs such as Met…
All models are wrong, some are useful: Model Selection with Limited Labels
Patrik Okanovic, Andreas Kirsch, Jannes Kasper +3
We introduce MODEL SELECTOR, a framework for label-efficient selection of pretrained classifiers. Given a pool of unlabeled target data, MODEL SELECTOR samples a small subset of hi…
Collaboratively Learning Federated Models from Noisy Decentralized Data
Haoyuan Li, Mathias Funk, Nezihe Merve Gürel +1
Federated learning (FL) has emerged as a prominent method for collaboratively training machine learning models using local data from edge devices, all while keeping data decentrali…
C-RAG: Certified Generation Risks for Retrieval-Augmented Language Models
Mintong Kang, Nezihe Merve Gürel, Ning Yu +2
Despite the impressive capabilities of large language models (LLMs) across diverse applications, they still suffer from trustworthiness issues, such as hallucinations and misalignm…