4 papers · 1 filter
Preservation of Language Understanding Capabilities in Speech-aware Large Language Models
Marek Kubis, PaweŠSkórzewski, Iwona Christop +4
The paper presents C3T (Cross-modal Capabilities Conservation Test), a new benchmark for assessing the performance of speech-aware large language models. The benchmark utilizes tex…
ClonEval: An Open Voice Cloning Benchmark
Iwona Christop, Tomasz KuczyÅski, Marek Kubis
We present a novel benchmark for voice cloning text-to-speech models. The benchmark consists of an evaluation protocol, an open-source library for assessing the performance of voic…
Polish-English medical knowledge transfer: A new benchmark and results
Åukasz Grzybowski, Jakub Pokrywka, MichaÅ CiesióÅka +2
Large Language Models (LLMs) have demonstrated significant potential in handling specialized tasks, including medical problem-solving. However, most studies predominantly focus on…
LLMzSzÅ: a comprehensive LLM benchmark for Polish
Krzysztof Jassem, MichaÅ CiesióÅka, Filip GraliÅski +5
This article introduces the first comprehensive benchmark for the Polish language at this scale: LLMzSzÅ (LLMs Behind the School Desk). It is based on a coherent collection of Pol…