75 citations · 394 across the 149 of their papers we have counts for
14 papers · 1 filter
WorldSpeech: A Multilingual Speech Corpus from Around the World
Antonis Asonitis, Luca A. Lanzendörfer, Frédéric Berdoz +1
Automatic speech recognition (ASR) performs well for high-resource languages with abundant paired audio-transcript data, but its accuracy degrades sharply for most languages due to…
Reasoning Boosts Opinion Alignment in LLMs
Frédéric Berdoz, Yann Billeter, Yann Vonlanthen +1
Opinion modeling aims to capture individual or group political preferences, enabling applications such as digital democracies, where models could help shape fairer and more popular…
EuroSpeech: A Multilingual Speech Corpus
Samuel Pfisterer, Florian Grötschla, Luca A. Lanzendörfer +2
Recent progress in speech processing has highlighted that high-quality performance across languages requires substantial training data for each individual language. While existing…
Text Compression for Efficient Language Generation
David Gu, Peter Belcak, Roger Wattenhofer
We challenge the prevailing assumption that LLMs must rely fully on sub-word tokens for high-quality text generation. To this end, we propose the "Generative Pretrained Thoughtform…
ACORD: An Expert-Annotated Retrieval Dataset for Legal Contract Drafting
Steven H. Wang, Maksim Zubkov, Kexin Fan +5
Information retrieval, specifically contract clause retrieval, is foundational to contract drafting because lawyers rarely draft contracts from scratch; instead, they locate and re…
Assessing Adversarial Robustness of Large Language Models: An Empirical Study
Zeyu Yang, Zhao Meng, Xiaochen Zheng +1
Large Language Models (LLMs) have revolutionized natural language processing, but their robustness against adversarial attacks remains a critical concern. We presents a novel white…
Exponentially Faster Language Modelling
Peter Belcak, Roger Wattenhofer
Language models only really need to use an exponential fraction of their neurons for individual inferences. As proof, we present UltraFastBERT, a BERT variant that uses 0.3% of its…
Examining the Emergence of Deductive Reasoning in Generative Language Models
Peter Belcak, Luca A. Lanzendörfer, Roger Wattenhofer
We conduct a preliminary inquiry into the ability of generative transformer models to deductively reason from premises provided. We observe notable differences in the performance o…
Cascaded Beam Search: Plug-and-Play Terminology-Forcing For Neural Machine Translation
Frédéric Odermatt, Béni Egressy, Roger Wattenhofer
This paper presents a plug-and-play approach for translation with terminology constraints. Terminology constraints are an important aspect of many modern translation pipelines. In…
Beyond Prompting: Making Pre-trained Language Models Better Zero-shot Learners by Clustering Representations
Yu Fei, Ping Nie, Zhao Meng +2
Recent work has demonstrated that pre-trained language models (PLMs) are zero-shot learners. However, most existing zero-shot methods involve heavy human engineering or complicated…
On Isotropy Calibration of Transformers
Yue Ding, Karolis Martinkus, Damian Pascual +2
Different studies of the embedding space of transformer models suggest that the distribution of contextual representations is highly anisotropic - the embeddings are distributed in…
A Plug-and-Play Method for Controlled Text Generation
Damian Pascual, Beni Egressy, Clara Meister +2
Large pre-trained language models have repeatedly shown their ability to produce fluent text. Yet even when starting from a prompt, generation can continue in many plausible direct…
Self-Supervised Contrastive Learning with Adversarial Perturbations for Defending Word Substitution-based Attacks
Zhao Meng, Yihan Dong, Mrinmaya Sachan +1
In this paper, we present an approach to improve the robustness of BERT language models against word substitution-based adversarial attacks by leveraging adversarial perturbations…
Towards BERT-based Automatic ICD Coding: Limitations and Opportunities
Damian Pascual, Sandro Luck, Roger Wattenhofer
Automatic ICD coding is the task of assigning codes from the International Classification of Diseases (ICD) to medical notes. These codes describe the state of the patient and have…