2 citations · 3 across the 4 of their papers we have counts for
11 papers
On Isotropy Calibration of Transformers
Yue Ding, Karolis Martinkus, Damian Pascual +2
Different studies of the embedding space of transformer models suggest that the distribution of contextual representations is highly anisotropic - the embeddings are distributed in…
A Plug-and-Play Method for Controlled Text Generation
Damian Pascual, Beni Egressy, Clara Meister +2
Large pre-trained language models have repeatedly shown their ability to produce fluent text. Yet even when starting from a prompt, generation can continue in many plausible direct…
BERT is Robust! A Case Against Synonym-Based Adversarial Examples in Text Classification
Jens Hauser, Zhao Meng, Damián Pascual +1
Deep Neural Networks have taken Natural Language Processing by storm. While this led to incredible improvements across many tasks, it also initiated a new research field, questioni…
Towards BERT-based Automatic ICD Coding: Limitations and Opportunities
Damian Pascual, Sandro Luck, Roger Wattenhofer
Automatic ICD coding is the task of assigning codes from the International Classification of Diseases (ICD) to medical notes. These codes describe the state of the patient and have…
Of Non-Linearity and Commutativity in BERT
Sumu Zhao, Damian Pascual, Gino Brunner +1
In this work we provide new insights into the transformer architecture, and in particular, its best-known variant, BERT. First, we propose a method to measure the degree of non-lin…
Directed Beam Search: Plug-and-Play Lexically Constrained Language Generation
Damian Pascual, Beni Egressy, Florian Bolli +1
Large pre-trained language models are capable of generating realistic text. However, controlling these models so that the generated text satisfies lexical constraints, i.e., contai…