14 citations · 32 across the 9 of their papers we have counts for
11 papers
A Variational AutoEncoder for Transformers with Nonparametric Variational Information Bottleneck
James Henderson, Fabio Fehr
We propose a VAE for Transformers by developing a variational information bottleneck regulariser for Transformer embeddings. We formalise the embedding space of Transformer encoder…
Multilingual Extraction and Categorization of Lexical Collocations with Graph-aware Transformers
Luis Espinosa-Anke, Alexander Shvets, Alireza Mohammadshahi +2
Recognizing and categorizing lexical collocations in context is useful for language learning, dictionary compilation and downstream NLP. However, it is a challenging task due to th…
What Do Compressed Multilingual Machine Translation Models Forget?
Alireza Mohammadshahi, Vassilina Nikoulina, Alexandre Berard +3
Recently, very large pre-trained models achieve state-of-the-art results in various natural language processing (NLP) tasks, but their size makes it more challenging to apply them…
Bag-of-Vectors Autoencoders for Unsupervised Conditional Text Generation
Florian Mai, James Henderson
Text autoencoders are often used for unsupervised conditional text generation by applying mappings in the latent space to change attributes to the desired values. Recently, Mai et…
Imposing Relation Structure in Language-Model Embeddings Using Contrastive Learning
Christos Theodoropoulos, James Henderson, Andrei C. Coman +1
Though language model text embeddings have revolutionized NLP research, their ability to capture high-level semantic information, such as relations between entities in text, is lim…
The DCU-EPFL Enhanced Dependency Parser at the IWPT 2021 Shared Task
James Barry, Alireza Mohammadshahi, Joachim Wagner +2
We describe the DCU-EPFL submission to the IWPT 2021 Shared Task on Parsing into Enhanced Universal Dependencies. The task involves parsing Enhanced UD graphs, which are an extensi…
Syntax-Aware Graph-to-Graph Transformer for Semantic Role Labelling
Alireza Mohammadshahi, James Henderson
Recent models have shown that incorporating syntactic knowledge into the semantic role labelling (SRL) task leads to a significant improvement. In this paper, we propose Syntax-awa…
Multi-Adversarial Learning for Cross-Lingual Word Embeddings
Haozhou Wang, James Henderson, Paola Merlo
Generative adversarial networks (GANs) have succeeded in inducing cross-lingual word embeddings -- maps of matching words across languages -- without supervision. Despite these suc…
Recursive Non-Autoregressive Graph-to-Graph Transformer for Dependency Parsing with Iterative Refinement
Alireza Mohammadshahi, James Henderson
We propose the Recursive Non-autoregressive Graph-to-Graph Transformer architecture (RNGTr) for the iterative refinement of arbitrary graphs through the recursive application of a…
Graph-to-Graph Transformer for Transition-based Dependency Parsing
Alireza Mohammadshahi, James Henderson
We propose the Graph2Graph Transformer architecture for conditioning on and predicting arbitrary graphs, and apply it to the challenging task of transition-based dependency parsing…
Partially-supervised Mention Detection
Lesly Miculicich, James Henderson
Learning to detect entity mentions without using syntactic information can be useful for integration and joint optimization with other tasks. However, it is common to have partiall…